Incubator Reports

Innovation for Digital Transformation and Policy Analytics

November 2022

AUTHORS: Dr. Azad M. Madni, Dr. Dan Erwin

Current system, product, and service validation processes encompass a limited number of ad hoc system-level scenarios in which the system, process, or service is evaluated against the stakeholders’ goals. However, budgetary and schedule constraints almost always limit validation scenarios, which reduces confidence in the validation process. In this study, we have developed a risk-based validation approach in which scenario selection is based on the importance and magnitude of the risk they can retire. We have presented an exemplar acquisition problem and attendant scenarios that can benefit from this approach. The exemplar problem is acquisition of a UAV-based forest fire detection system. This problem was selected because it is sufficiently rich in terms of scenario modifiers and sufficiently complex to demonstrate our innovative risk-based validation approach.  

From the behaviors and performance expected of a system, the researchers tallied the functions and performance values required for success. The metamodel helped determine what validation events must be performed, the rigor needed, the resources required, and how validation events can be minimized and still cover the system validation space. 

In a potential Phase II, we will implement this methodology and demonstrate its efficacy and effectiveness on an acquisition problem of interest to AIRC and its customers. We intend to transition the software to Jet Propulsion Laboratory, a transition site, and any other AIRC recommended transition sites.