Utilizing Uncertainty Information in Early Model
Validation
Magnus Carlsson*, Sören Steinkellner†, Hampus Gavel‡
Saab Aeronautics, Linköping, Sweden, SE-581 88
and
Johan Ölvander§
Linköping University, Linköping, Sweden, SE-581 83
This paper proposes a pragmatic approach enabling early model validation activities
with a limited availability of system level measurement data. The method utilizes
information obtained from the common practice of component validation to assess
uncertainties on model top level. Focusing on industrial applicability, the method makes
use of information normally available to engineers developing simulation models of
existing or not yet existing systems. This is in contrast to the traditional sensitivity
analysis requiring the user to quantify component parameter uncertainties – a task
which, according to the authors’ experience, may be far from intuitive. As the proposed
method enables uncertainties to be defined for a component’s outputs (characteristics)
rather than its inputs (parameters), it is hereby termed output uncertainty. The method is
primarily intended for use in large-scale mathematical 1-D dynamic simulation models
of physical systems with or without control software, typically described by Ordinary
Differential Equations (ODE) or Differential Algebraic Equations (DAE).
It is shown that the method may result in a significant reduction in the number of
uncertain parameters that require consideration in a simulation model. The uncertainty
quantification of these parameters also becomes more intuitive. Since this implies a
substantial improvement in the conditions of conducting sensitivity analysis or
optimization on large-scale simulation models, the method facilitates early model
validation. In contrast to sensitivity analysis with respect to a model’s original
component parameters, which only covers one aspect of model uncertainty, the output
uncertainty method enables assessment also of other kinds of uncertainties, such as
uncertainties in underlying equations or uncertainties due to model simplifications. To
increase the relevance of the method, a simulation model of a radar liquid cooling system
is used as an industrial application example.
I.
Introduction
M
ODELING and simulation (M&S) has been used in the aerospace industry for many years and there is an
ongoing trend to further increase the portion of M&S. This is highly visible in the CRESCENDO1 project,
in which methodologies and tools enabling collaborative design, Virtual Testing (VT) and Virtual Certification
(VC) are developed. One fundamental purpose of the increased usage of M&S is to reduce the amount of
physical prototyping and physical testing – activities which normally demand a significant amount of resources.
Related to this is the aim to further enhance the ability to take early model-based design decisions, as well as
enhancing the ability to use M&S as a support in certification of aircraft systems. A necessary condition to
achieve this is to be able to answer questions such as To what extent can we trust the model? or How well does
the model represent the real system? or Does the model cover the intended use?
The above questions deal with model validation and, in the broader scope, M&S credibility. Depending on
the model complexity and the intended use, performing a relevant assessment of a model’s credibility might be a
challenging task. Substantial research has been done in this field, proposing different methods for making
assessments of model credibility. Three examples are the Credibility Assessment Scale (CAS) proposed in the
NASA Standard for Models and Simulations2, the Predictive Capability Maturity Model (PCMM) proposed by
*
M.Sc., Systems Engineer, Modeling and Simulation, Vehicle Systems
Tech. lic., Systems Engineer, Modeling and Simulation, Vehicle Systems
‡
Dr., Manager, Modeling and Simulation, Vehicle Systems
§
Prof., Division of Machine Design, Department of Management and Engineering
1
American Institute of Aeronautics and Astronautics
†