Research Reports

Test and Evaluation Methods for Middle-Tier Acquisition – Option Year 1

PUBLIC RELEASE
January 2025

COMPLETED
September 2024

AUTHORS:  Dr. Laura Freeman, Mr. Geoffrey Kerr
VIRGINIA TECH NATIONAL SECURITY INSTITUTE

This report is a culmination of research activities conducted across seven organizations in support of the Director, Operational Test and Evaluation’s (DOT&E) Strategic Initiatives, Policy, and Emerging Technologies Directorate (SIPET). The methods developed in this research reflect emerging technologies, a changing threat landscape, and the need for efficient and effective test and evaluation (T&E) to ensure capabilities delivered to warfighters work as intended.

This report builds on the foundational work performed in the base year 1 by AIRC in partnership with the DOT&E. The research captures current best
practices for improving T&E. The research looked at improving practices through policy, exemplar tools, training material, and the transition of emerging technology into practical application by T&E professionals. The research efforts were conducted as authorized by the execution of the option year contracts for WRT-1070: Test and Evaluation Methods for Middle Tier Acquisition (MTA) and WRT-1071: Digital Transformation in Test and Evaluation. The research team aligned their efforts to three of the DOT&E Implementation Plan (I-Plan) pillars: Test the Way We Fight, Accelerate the Delivery of Weapons That Work, and Pioneer T&E of Weapon Systems Built to Change Over Time.

The team organized their support to these pillars in the following lines of concentration/effort:

• Joint Test Concept (JTC) – Develop methodologies to test systems in order to support the operational assessment of system-of-system joint operations.
• Integrated Testing – Applying statistical methods and leveraging contractor testing, development testing, and operational testing (OT) to more efficiently perform operational assessments.
• Digital Engineering – Applying modern modeling techniques to integrate model-based test planning, modeling and simulation, test execution, model-based systems engineering, and digital product lifecycle management to ensure scientific rigor is ensured and efficient test planning and execution are realized on legacy platform development and born-digital weapon system development.
• Artificial Intelligence/Machine Learning (AI/ML) and T&E – Leverage state-of-the-art AI/ML techniques to accomplish T&E of Department of Defense (DoD) systems and develop ethical and responsible methods for performing evaluation of weapon systems that are enabled by AI/ML technologies.

In addition to the above pillar-specific efforts, the research team partnered with DOT&E to revise the current T&E policy by providing technical consultation, reviewing and editing six policy documents, and planning for policy companion guide development for each of the new policy documents to help transition T&E practitioners to adapt their practices to be consistent with the new policy.

A related AIRC report involving this research is Digital Transformation in Test and Evaluation for AI/ML, Autonomous, and Evolving Systems – Option Year 1.