Representing model inadequacy: A stochastic operator approach
Rebecca E. Morrison, Todd A. Oliver, Robert D. Moser
April 7, 2016
arXiv:1604.01651v1 [cs.CE] 6 Apr 2016
Abstract
Mathematical models of physical systems are subject to many uncertainties such as measurement
errors and uncertain initial and boundary conditions. After accounting for these uncertainties, it is often
revealed that discrepancies between the model output and the observations remain; if so, the model is
said to be inadequate. In practice, the inadequate model may be the best that is available or tractable,
and so despite its inadequacy the model may be used to make predictions of unobserved quantities. In
this case, a representation of the inadequacy is necessary, so the impact of the observed discrepancy
can be determined. We investigate this problem in the context of chemical kinetics and propose a new
technique to account for model inadequacy that is both probabilistic and physically meaningful. A
stochastic inadequacy operator S is introduced which is embedded in the ODEs describing the evolution
of chemical species concentrations and which respects certain physical constraints such as conservation
laws. The entries of S are governed by probability distributions, which in turn are characterized by a set
of hyperparameters. The model parameters and hyperparameters are calibrated using high-dimensional
hierarchical Bayesian inference. We apply the method to a typical problem in chemical kinetics— the
reaction mechanism of hydrogen combustion.
1
Introduction
Model inadequacy is a complex and critical issue that affects nearly all realms of computational science
and engineering. In general, models of physical systems are imperfect: they rely on abstractions and simplifications which do not perfectly represent the modeled system. Sometimes the imperfections are small
enough that any discrepancy between the model and reality is dominated by observation error, such that
the discrepancy is essentially undetectable given existing measurement technology. In contrast, a model is
demonstrably inadequate when the imperfections lead to a detectable inconsistency between the model and
observations. Such inadequacies are often detected during model validation, which is the process of assessing
whether a given mathematical model, including representations of relevant uncertainties, is consistent with
knowledge regarding the modeled system [3, 4, 41]. When an inadequacy is detected, one would generally
prefer to improve the model to remove the discrepancy prior to using it to make predictions, but such improvement is often not feasible. In this case, a representation of the inadequacy is needed to quantify the
possible impact of model inadequacy on computed quantities of interest. However, the formulation of such
model inadequacy representations is an open problem. In this work, we propose a new technique to account
for model inadequacy in the context of chemical kinetics models. Specifically, the model inadequacy in the
equations describing the evolution of the chemical system is represented by a stochastic operator which is
specifically constructed to respect certain non-negotiable constraints on the system. This approach results
in a model inadequacy representation that is both probabilistic and physically meaningful. The method is
demonstrated on a typical problem in chemical kinetics, namely the reaction mechanism model for hydrogen
combustion, where it is shown that the formulation is flexible enough to account for significant inadequacies
present in the original model.
Chemical mechanisms and kinetics models describe the process and rates of chemical reactions [49, 53].
In general, a reaction mechanism is extraordinarily complex, even when there are only two or three initial
reactants. An accurate description of the chemical processes involved in the oxidation of hydrocarbons, for
1
example, may include hundreds or thousands of reactions and fifty or more chemical species [46, 51]. At the
same time, there is significant uncertainty in the reaction rates for these reactions; recent efforts to address
this include [32, 36, 9]. Furthermore, kinetics models of these chemical mechanisms are commonly embedded
within a larger fluids calculation to represent combustion. The chemical dynamics must then be represented
at every point in space and time. Because the computational cost of such detailed mechanisms is so high,
it is common practice to use drastically reduced mechanisms. Such reduced models are commonly found in
the combustion literature [11, 51, 54]. However, errors introduced by the reduced models may render the
model inadequate even if the detailed model it is based on is not. This work is concerned with accounting
for the inadequacy resulting from the use of a reduced chemical mechanism.
If a chemical kinetics model is inadequate, it would be best to improve the kinetics model directly to
eliminate the inadequacy. Indeed, refinement of chemical mechanisms in combustion is an active topic of
research; for a small sample focused on H2 /O2 reactions, see [10, 11, 15, 40]. However, this type of refinement
is commonly not an option, because of a lack of physical insight required to develop a higher fidelity model,
a lack of detailed observational data to support such model development, or a lack of time or other resources
required for the model development process. Further, even when a higher fidelity model exists, a lack of
computational resources necessary to obtain solutions of that model may make it impractical to use. Thus,
there is a general need for methods that account for the discrepancy between the inadequate model and
the observations that do not require traditional model improvement. To develop such a representation, one
must adopt a mathematical framework for reasoning about the model inadequacy. In this work, we adopt
a Bayesian point of view [17, 29], and thus, any lack of knowledge—and model inadequacy specifically—is
modeled using probability. Further, the Bayesian approach offers a natural framework for representing all
uncertainties that arise in using reduced kinetics models to make predictions—including modeling inadequacy
of course but also uncertain kinetics parameters and measurement uncertainties—as well as a natural method
for updating these representations to incorporate information from data [14, 30, 12, 45].
In the Bayesian statistics community, a common approach to representing model inadequacy is to pose
and calibrate a purely statistical model of the discrepancy between the model output and the true value of
that output. This approach was pioneered by Kennedy and O’Hagan [31] and has been used widely and
successfully in many applications [4, 27, 28, 50]. However, the structure of the Kennedy and O’Hagan representation precludes its use for characterization of the impact of inadequacy on predictions of an unobserved
quantity [41], which is often the purpose of a model, particularly in science and engineering applications. Further, we generally have both qualitative and quantitative information about the phenomena being modeled,
which imposes constraints on the inadequacy representation. Exploiting this information should lead to a
more reliable formulation, especially for making predictions. Imposing these physical constraints necessarily
requires that the structure of the inadequacy representation depends on the physical problem.
In this work, inadequacy representations are formulated not as corrections of the model output, but
as stochastic enrichments of the model itself which are specifically constructed to respect known physical
constraints. A set of chemical reactions is modeled by describing the time derivative of the species’ concentrations and temperature, such that the model consists of a set of nonlinear ordinary differential equations.
The necessary constraints in this context are conservation of atoms, conservation of energy, and nonnegativity of concentrations. The model inadequacy representation developed here is characterized by a stochastic
operator S, which is added to the formulation of the time derivatives of the species’ concentrations and
temperature. This operator is constructed such that all realizations of the solution of the enriched reduced
model—i.e. the reduced model coupled with the inadequacy representation—conserve atoms and energy and
have nonnegative concentrations at all times. The main component of the operator is additive, linear, and
probabilistic, encoded in a random matrix S. The use of the term random matrix implies that each entry is
characterized by a probability distribution. This is consistent with the definition of random matrices from
random matrix theory (see [19, 35]), although in that field a random matrix is usually much less constrained
than in the present case, and its properties (such as the distributions of the eigenvalues) are found in the
limit as the size of the matrix goes to infinity. A few applications of random matrices to engineering problems
are presented by Soize [47, 48]. However, this work also differs from our approach in that the probability
of a given matrix is characterized by properties of the entire matrix, such as the determinant, whereas we
2
consider the independent distributions of each nonzero entry. Moreover, the inadequacy operator S may include more general effects in addition to or instead of the random matrix, such as nonlinearities or multiple
matrices.
The parameters in the stochastic inadequacy operator S are uncertain because the exact form of the
inadequacy is unknown. Because of this lack of knowledge, the parameters of the inadequacy representation
are characterized by probability distributions, yielding a stochastic forward problem. Each distribution is
characterized by a corresponding set of hyperparameters. Thus, to fully describe the inadequacy model, it is
necessary to infer these hyperparameters. And, since the inadequacy model is calibrated simultaneously with
the reduced model, the task is now to infer the reduced model kinetics parameters and the hyperparameters
of the inadequacy model. We use a hierarchical Bayesian approach [6] to formulate this inverse problem.
After calibration, the process of validation checks that the model output is consistent with the observations.
The particular approach used here is essentially posterior predictive assessment [22, 44], and is described
in the context of physical models in [39]. Specifically, the validity of the model is judged based on the
plausibility of observations according to the uncertain model.
The rest of the paper is organized as follows. In § 2, we give a brief overview of kinetics modeling. In § 3,
the general formulation and properties of the stochastic operator are presented. § 4 describes the Bayesian
framework for calibration and validation of the various models, including hierarchical Bayesian modeling and
validation under uncertainty. In § 5, the approach is applied to the specific case of hydrogen combustion.
Concluding remarks are given in § 6.
2
Chemical mechanism models
Chemical mechanisms and kinetics models describe the process and rates of chemical reactions. In a typical
chemical reaction, there is a set of reactant species which, after a complex series of intermediate reactions,
ultimately form the chemical products. These intermediate steps, in which chemical species react directly
with each other, are called elementary reactions. The set of elementary reactions is called the reaction
mechanism, and a typical combustion problem may include tens to thousands of elementary reactions. This
section provides a minimal introduction to the essentials of chemical kinetics necessary to understand the
development of the model inadequacy representation in § 3. For more details on chemical kinetics, see [49]
for a general text and [53] for a presentation focused on combustion.
To introduce the main concepts, consider the following example reaction set with four species and two
reversible (meaning that the reactions can proceed in either directions—e.g., from A and B to C or from C
to A and B) elementary reactions:
kf
1
−
*
A+B−
)
−
−C
k1b
k2f
−
*
A+C−
)
−
− D,
k2b
where A, B, C, and D denote the different chemical species, and k1f , k2f and k1b , k2b are the forward and
backward rate coefficients, respectively. Let x = [x1 , x2 , x3 , x4 ]T be the vector of molar concentrations
(having dimensions moles per unit volume) corresponding to species A, B, C, D. In the differential equation
describing the evolution of the species’ concentrations, each elementary reaction is a sink for the reactant
species and a sources for the product species. The rate of each reaction is often modeled as linear in the
concentration of the reactants, although this power, or order, associated with a given species may be nonunity. With the assumption that it is linear in each species, the forward rate expressions of the two reactions
are thus
r1f = k1f x1 x2
(1)
r2f
(2)
=
k2f x1 x3 .
3
Similarly, the backward rates are given by
r1b = k1b x3
(3)
r2b = k2b x4 .
(4)
Finally, the ODEs for the molar concentrations are
ẋ1 = −r1f + r1b − r2f + r2b
ẋ2 =
ẋ3 =
ẋ4 =
−r1f
+r1f
+r2f
+
r1b
−
r1b
−
(5)
(6)
−
r2f
+
r2b
r2b .
(7)
(8)
The rate coefficients k are generally functions of temperature, and may follow a given empirical form
depending on the specific reaction. A common form is the Arrhenius Law,
k(T ) = Ae−E/R
◦
T
,
(9)
for some prefactor A, activation energy E, and universal gas constant R◦ . Another common form is the
modified Arrhenius,
◦
k(T ) = AT b e−E/R T ,
(10)
with the additional constant b. The Arrhenius or modified Arrhenius forms are often found in the literature to
describe the forward rate coefficient, and this is true in the example problem in § 5. However, the backwards
rate coefficients are usually not specified. Instead, these are determined from the equilibrium constant which
depends only on the thermodynamics of the reaction.
Given a reversible reaction and the forward rate coefficients, there are various software libraries which
will solve for the backwards rate coefficients. In this work, a chemistry software library called Antioch (A
New Templated Implementation of Chemistry Hydrodynamics) was used to set up the chemical model, query
thermodynamic information, and solve for the reverse reaction rates [1].
To complete the specification of the system, a governing equation for temperature is required. Here
chemical reactions occuring at constant volume are considered, and so the evolution of the temperature
modeled through an energy conservation equation. Temperature changes occur due to the difference in
chemical energy between the reactants and products. To model this, we start by describing the internal
energy U of an ideal gas. In an ideal gas, the internal energy depends only on temperature (not v or p) and
the species’ concentrations:
X
U (T, x) =
ui (T )xi ,
(11)
i
and so
X ∂ui
∂xi
dU
=
xi + ui
dt
∂t
∂t
i
X ∂T
∂xi
=
cvi
xi + ui
∂t
∂t
i
X
∂T
∂xi
=
cvi xi + ui
.
∂t i
∂t
(12)
(13)
(14)
But since volume is constant, no work is done on the system, and the change in energy U is zero. Setting
4
dU/dt to zero and solving for dT /dt yields
dT
=−
dt
1
c
i v i xi
P
!

X

ui ẋi  .
(15)
i
Note that dT
dt is a function of both the molar concentrations and their time derivatives. The remaining
quantities cv (or cp ) and u (or h) are found in the literature along with entropy s. Usually, a set of seven
to nine constants are given which specify each quantity as a function of temperature. Standard NASA
polynomials are used in this work [34].
With the time derivative of temperature, the mathematical model of the reaction mechanism is complete.
To summarize, there is an ODE for the time derivative of each species and also for temperature. These can
be written more compactly as
F(x, ẋ, T ) = [ẋ1 , ẋ2 , . . . , ẋn , Ṫ ]T
(16)
where F is a general nonlinear operator consisting of the time derivatives of x and T . Note that F depends
on ẋ only through the energy equation.
3
Formulation of the model inadequacy
In contrast to the simple example mechanism in § 2, mechanisms describing complex chemical systems like
those encountered in combustion often include hundreds of reactions. One of the standard references for
methane combustion, for example, includes fifty-three species and 325 reactions [46]. Such models are often
referred to as detailed mechanisms and will be written here as D(xD , ẋD , T ). In the context of a reacting
flow simulation, such a large mechanism may be too computationally expensive to be practical, and thus it
is common to use a reduced chemistry model consisting of a subset of the species and reactions from the
detailed model.
To be concrete, suppose that the detailed model includes nD species and mD reactions. Then, the reduced
model includes nR species and mR reactions, where nR ≤ nD and mR < mD . The reduced model always
contains fewer reactions than the detailed; the number of species included in the reduced model may or may
not be smaller, although in practice it is almost always true that nR < nD . The reduced model is denoted
R(xR , ẋR , T ).
In the case that the reduced model does not adequately represent the detailed (or the real chemical reaction), one has two options: (1) improve the reduced model directly with more chemistry, or (2) incorporate
a representation of the model error of the reduced model. As noted in § 1, it is often impractical to improve
the model and thus, the focus of this work is on developing a generally applicable model inadequacy representation for reduced chemistry models. The formulation takes the form of a stochastic inadequacy operator
S, which is appended to the reduced model R, as indicated in Figure 1, which shows the progression from
the detailed model to the proposed stochastic model.
In the figure, the D and R operators correspond to the deterministic detailed and reduced models introduced above. The operator RG denotes the reduced mechanism used with a general stochastic inadequacy
model, Gω , where ω is a set of random variables. This could be, for example, a Gaussian process. On the
other hand, this term could represent an augmentation of the chemical mechanism obtained by incorporating
more reactions from the detailed model. In this case, the model inadequacy representation would in fact
be deterministic (with ω = {}). This approach would necessitate more information about the true chemical
reaction than we expect to have or are willing to use and thus is not generally applicable.
Here we take an approach which is intermediate between the strictly statistical representation of a Gaussian process model, and the strictly physics-based approach of incorporating more species and reactions. We
insist that the inadequacy representation respect all that we know about the model and why it is inadequate,
and make it stochastic because this information is incomplete implying that there are a range of consistent
behaviors, depending on the scenario. In particular, in the chemical kinetics case, we know that errors in the
5
Detailed model D
Reduced model R
order reduction
D(xD , ẋD , T )
R(xR , ẋR , T )
stochastic
inadequacy
formulation
calibration
data
(RS + S)(xS , ẋS , T )
parameterization,
physical constraints
RG + G ω
General inadequacy
model G
Operator model
O = RS + S
Figure 1: Relationship between the detailed model, deterministic reduced model, reduced model with general
model inadequacy representation, and reduced model with stochastic operator model inadequacy representation.
model are due to the fact that not all species are included in the representation, and that not all the possible ways (reactions) by which species transform into each other are represented. Further, both atoms and
energy must be conserved and species concentrations and temperature must remain positive. To account
for the effects of the missing species and reactions and to satisfy these constraints, a stochastic operator
representation of the inadequacy is posed, as shown on the bottom left of Figure 1. The reduced model RS
plus the stochastic inadequacy operator S is called the operator model O.
For clarity of notation, the superscripts (D, R, G S) on either the state vector x or the reduced model
R are included to reflect the model at hand. As will become clear, to satisfy the requirements mentioned
above, the inadequacy formulation alters the state vector x and the reduced model operator R, so xS and
RS differ from the corresponding xR and R.
3.1
3.1.1
Components and State of the Operator Model
Components of the operator
The main action of the inadequacy operator S is to modify the time derivatives of the species’ concentrations.
The operator consists of three pieces: a random matrix S, a non-linear operator A, and a non-linear operator
W.1 The random matrix S is intended to represent the most general linear correction that can be made to
the reduced chemistry model. However, to preserve an important property of the solution (see § 3.4), it is
necessary to introduce a non-linear modification, expressed as a non-linear operator A. The final role of the
operator is to properly account for energy changes by modifying the time derivative of temperature, encoded
in the non-linear operator W.
Note that S and A act on just the concentrations, while W acts on the concentrations and their derivatives.
Moreover, it is convenient to formulate S in terms of atomic concentrations, while Ŝ denotes the corresponding
matrix in terms of molar concentrations. This section focuses on S instead of Ŝ because many of the properties
1 In general, a script letter refers to a non-linear operator, capital letters to linear operators (matrices), lowercase bold letters
to vectors, and lowercase (unbolded) letters to scalars.
6
of the matrix (such as non-positivity of eigenvalues) are better expressed in terms of atoms instead of moles
(see § B). To map between S and Ŝ, the vector l is used, whose ith entry counts the number of atoms (of all
types) in one molecule of the ith species. For example, if the set of species is H2 , O2 , H, O, OH, H2 O, then
l = [2, 2, 1, 1, 2, 3]. Let L denote the nS × nS matrix with the entries of l on the diagonal. Then Ŝ = L−1 SL
applies to molar concentrations. Finally, putting the three pieces together,
S = Ŝ + A + W
−1
=L
SL + A + W,
(17)
or more explicitly,
S(xS , ẋS , T ) = L−1 SLxS + A(xS ) + W(xS , ẋS , T ).
3.1.2
(18)
(19)
Augmentation of the state vector
The reduced model tracks fewer species than the detailed model. It should be possible then, for the inadequacy formulation to represent this difference. However, we do not want to include all the extra (missing)
species in the inadequacy representation. Therefore, in order to account for the missing species in the reduced model, the state space is augmented by entries for all types of atoms. These entries are referred to as
catchall species and act as a sort of pool of each atom type. The precense of the catchall species allows the
operator S to move atoms to and from these pools instead of constraining every atom to one of the species
of the reduced model, which is overly restrictive because in the detailed model, atoms may move to species
that are not part of the reduced model. Thus, xS is of length nS = nR + nα , where nα is the number of
atom types, and is of the form
xS = [x1 , . . . , xnα , xnα +1 , . . . , xnα +nR ]T .
In general, we denote the catchall species of element X by X0 . For example, consider a reduced model
species that includes H2 , O2 , OH, and H2 O. Then the catchall species are H0 and O0 , and
xS = [x1 , x2 , x3 , x4 , x5 , x6 ]T
(20)
where x1 , . . . , x6 corresponds to H0 , O0 , H2 , O2 , OH, and H2 O, in that order.
This brings us to an important point about the structure of the reduced model: it takes on a different
form when used in conjunction with the stochastic operator S. There are two reasons for this. First, R now
acts on a vector space of dimension nS , although it has no effect on the first nα entries of xS . Second, the
effect of the catchall species on the energy equation is not additive. Because of this, the differential equation
for T is removed from R, and the entire calculation is accounted for with W. These changes are reflected
by writing the reduced model as RS when used with the stochastic operator S. That is, RS only includes
the differential equations for the concentrations as in the reduced model, and dT /dt is given entirely by W.
A natural question at this point is: what is the most general form of such a matrix S? We show that
the general structure includes many identically zero entries and is the result of enforcing two important
constraints: conservation of atoms, and non-negativity of concentrations. The matrix also exhibits some
interesting properties, including: 1) the columns sum to zero, 2) the diagonal is negative, 3) the matrix is
weakly diagonally dominant, and 4) the eigenvalues are non-positive. These properties are implied by the
physical constraints on the system and are discussed further in § 3.2 and Appendix B
3.2
Physical constraints and their implications
There are two non-negotiable constraints that our system must respect: (I) conservation of atoms, and (II)
non-negativity of concentrations. This ensures that the inadequacy operator respects physical laws that are
known to be true for the systems of interest.
7
3.2.1
Conservation of atoms
To enforce (I), first let E = [eij ] be the nα × nS matrix, where eij is the fraction of atoms of type i in one
molecule of species j. Then it must be that ES = 0. This ensures that the operator acts on all atoms of a
given type without creating or destroying atoms. That is, the ith row of E applied to the jth column of S
equal to 0 shows that the ith type of atom is conserved as the operator redistributes that atom type from
species j among the rest of the species.
To continue the example shown in § 3.1, consider the case with atom types H and O, and species H0 , O0 ,
H2 , O2 , OH, H2 O. Then matrix E takes the form:
1 0 1 0 1/2 2/3
E=
.
(21)
0 1 0 1 1/2 1/3
To satisfy the constraint that ES = 0, the matrix S is decomposed as:
S = CP,
(22)
where C is a deterministic matrix and P is probabilistic. The matrix C ensures conservation of atoms,
and P ensures that the concentrations are non-negative. The columns of C span the nullspace of E, i.e.
span(C) = null(E). Thus,
ES = ECP = (EC)P = 0 · P = 0.
(23)
E is of dimension nα × nS , so the dimension of the nullspace is nS − nα = nR . Thus C is of dimension
nS × nR and P is of dimension nR × nS .
3.2.2
Non-negativity of concentrations
The second constraint (II) is that the concentrations must not be negative. To see how to enforce this,
consider the differential equation for species Xi 2 :
ẋi = (RS (x, T ))i + (S(x))i
−1
S
= (R (x, T ))i + (L
(24)
SLx)i + (A(x))i .
(25)
We must ensure that xi ≥ 0 when xi = 0 for i = 1, . . . , nS . The first term of the RHS of (25) is not a
problem, as this is the non-linear part from the reaction mechanism and is thus already physically consistent
[20]. The same argument holds for A(x); more will be said about this in § 3.4. Note the the energy operator
W is not written above because it does not modify the derivative of xi .
Finally, the second term must satisfy the constraint. Although it is in terms of molar concentrations, it
is helpful to rephrase this in terms of atomic concentrations. That is, the constraint is satisfied for moles if
and only if it is satisfied for atoms:
L−1 SLx ≥ 0
⇐⇒
Sy ≥ 0.
(26)
sij lj xj
(27)
To prove this, consider the ith entry of L−1 SLx:
(L−1 SLx)i = L−1
X
j
=
2 We
1X
sij lj xj
li j
drop the superscript S from x here for ease of notation.
8
(28)
but lj xj = yj and all li > 0, i = 1, . . . , nS . Thus,
X
1X
sij lj xj ≥ 0 ⇐⇒
sij yj ≥ 0.
li j
j
(29)
But the final term is exactly the ith element of Sy.
To continue in terms of atomic concentrations yi :
(Sy)i = sii yi +
X
sij yj .
(30)
j6=i
The first term from the diagonal, sii yi , automatically respects the constraint: sii may be set to be any
constant
value, since then sii yi → 0 as yi → 0. To enforce the constraint, it must be that the sum,
P
j6=i sij yj , is greater than zero. But this sum does not depend on yi , so we choose to set sij ≥ 0 for
all i 6= j. This could
P be made less restrictive by incorporating information from the non-linear system,
i.e. set (RS (x))i + j6=i ŝij xj ≥ 0, but this would violate the linearity assumption on S. It would also
necessitate using information from the reduced model, whereas we aim to constrain the inadequacy operator
independently of R.
3.2.3
Sparsity of S
In practice, many of the entries of S are identically zero. In theory, S could be completely dense if every
species included every type of atom. However, this never occurs in practical combustion reactions. The
following proves which entries of S are identically zero, using an argument based on the zeros of the matrix
E.
Theorem 3.1. Consider the ith row of E. Let Ji = {j|eij 6= 0} and Jic = {j|eij = 0}. Then every element
sjk = 0 for j ∈ Ji and k ∈ Jic .
Proof. Consider the ith row of E and the kth column of S. We have
X
0=
eij sjk
(31)
j
=
X
eij sjk +
X
eij sjk
(32)
j∈Jic
j∈Ji
=
X
eij sjk + 0.
(33)
j∈Jic
But since j and k are in disjoint sets, the sum in line (33) does not include the diagonal term sjj . But the
diagonal term is the only negative value in the k column. Thus, all sjk = 0, where j ∈ Ji and k ∈ Jic .
For another method for determining the sparsity of S, see Appendix B. In addition to sparsity, the
constraints on S imply that it has non-positive eigenvalues. See Appendix B for more details.
3.3
Construction of the matrix S
The structure of S is now clear; the next step is to actually construct it. The challenge here is that both
constraints must be simultaneously satisfied by any realization of S. This subsection presents a method
for construction of the operator. To help demonstrate the upcoming matrix decompositions and inequality
constraints, the construction will also be shown for the example set of species (H0 , O0 , H2 , O2 , OH, H2 O).
9
In this case, S has the form

s1,1
 0

s3,1
S=
 0

 0
0
0
s2,2
0
s4,2
0
0
s1,3
0
s3,3
0
0
0
0
s2,4
0
s4,4
0
0
s1,5
s2,5
s3,5
s4,5
s5,5
s6,5

s1,6
s2,6 

s3,6 
,
s4,6 

s5,6 
s6,6
(34)
where the diagonal elements are non-positive and the off-diagonal elements are non-negative. Here, nR = 4,
nα = 2.
First, C is formed as follows: let the bottom nR × nR block be the identity matrix InR . The remaining
top nα rows will be the negative of the last nR columns of E. Let this matrix block be denoted E ∗ . Note
that every element of E ∗ is non-positive. So, C has the form
#
"
E∗
C=
.
(35)
InR
As seen earlier for this example,
1
E=
0
so
−1
E =
0
∗
and
0
1

−1
0

1
C=
0

0
0
1
0
0
−1
0
−1
0
1
0
0
1/2
1/2
2/3
,
1/3
(36)
−1/2
−1/2
−2/3
−1/3
(37)

−2/3
−1/3

0 
.
0 

0 
1
(38)
0
1
−1/2
−1/2
0
0
1
0
Next, P is an nR × nS random matrix. To construct P , the first step is to specify which entries are
non-negative, non-positive, or strictly zero. Then, by taking advantage of the special structure of C, it is
possible to transfer the inequalities placed on the entries of S to those of P . Let P1 contain the first nα
columns of P , and P2 the remaining nR columns. So far we have
S = CP
"
#
E∗ P1 P2
=
InR
#
"
E ∗ P1 E ∗ P2
=
InR P1 InR P2
"
#
E ∗ P1 |E ∗ P2
=
.
I nR P
(39)
(40)
(41)
(42)
The bottom row of (42) shows how to transfer the inequalities from matrix S to P . Since P is left-multiplied
by the identity matrix, it must be that the signs match for the corresponding elements of S. In particular,
10
for 1 < i ≤ nR and ∀j, then
pi,j ≤ 0 if s(i+nα ),j ≤ 0
(43)
pi,j ≡ 0 if s(i+nα ),j ≡ 0.
(45)
pi,j ≥ 0 if s(i+nα ),j ≥ 0
(44)
Thus, in the example,

p1,1
 0
P =
 0
0
0
p2,2
0
0
p1,3
0
0
0
0
p2,4
0
0
p1,5
p2,5
p3,5
p4,5

p1,6
p2,6 
,
p3,6 
p4,6
(46)
where
p1,3 , p2,4 , p3,5 , p4,6 ≤ 0
and
p1,1 , p1,5 , p1,6 , p2,2 , p2,5 , p2,6 , p3,6 , p4,5 ≥ 0.
Note that the number of non-zero elements in P is 12.
The three inequalities above (43-45) are necessary but not sufficient as this only guarantees the inequalitites of the bottom row of (42) hold. The top row introduces more restrictive inequalities on a subset of
the entries of P . First consider the top left block. The only nonzero elements here are the negative entries
on the diagonal. There can be no non-zero off-diagonal elements of S in this block, because each row and
column correspond to a catchall species, and atoms can never move from one catchall to another because
they are of different types, by definition. But all the entries of E ∗ are non-positive, and all entries of P1 are
non-negative by (44) (these correspond to off-diagonal elements of S). Thus, the diagonal elements of S in
this top left block are guaranteed to be non-positive, as required.
Lastly, consider the top right block: E ∗ P2 . To guarantee that these elements are non-negative, it is
necessary that the negative entries of P2 (on its diagonal) are large enough in magnitude. For these elements
si,k in the top right block, 1 ≤ i ≤ nα and nα < k ≤ ns . Now
∗
0 ≤ si,k = E(i,·)
P(·,2k )
=
=
=
(47)
∗
E(i,·)
P(·,k+nα )
∗
E(i,·) P(·,k0 )
(48)
X
(50)
e∗i,j pj,k0 ,
(49)
j
where k 0 = k + nα . The only positive term above in the sum is e∗i,k pk,k0 , so this implies
e∗i,k pk,k0 ≥
X
e∗i,j pj,k0 .
(51)
j6=k
A similar inequality is placed on the each element pk,k0 for each type of atom (each row of E ∗ that multiplies
the k 0 th column of P ). Therefore, to complete the set of inequalities on P , it is sufficient that, for i = 1, . . . , nα
and k = 1, . . . , nR :
X
1
− pk,k0 ≥
max(e∗i,j )pj,k0 ,
(52)
i
mini (e∗i,k )
j6=k
or, in terms of the matrix C:
− pk,k0 ≥
X
1
max |ci,j |pj,k0 .
i
mini |ci,k |
j6=k
For use in the following development, denote the RHS of (53) above as qk0 .
11
(53)
In the example, the extra constraints from E ∗ P2 correspond to the diagonal elements of P : p1,3 , p2,4 ,
∗
∗
p3,5 , p4,6 . For example, the constraint s1,5 ≥ 0 implies E(1,·)
P(·,5) ≥ 0 and s2,5 ≥ 0 implies E(2,·)
P(·,5) ≥ 0.
These two constraints are then
1
−1p1,5 − 0p2,5 − p3,5 −
2
1
−0p1,5 − 1p2,5 − p3,5 −
2
2
p4,5 ≥ 0
3
1
p4,5 ≥ 0.
3
(54)
(55)
The two lines above can be condensed into a single inequality which is stronger than either of the two as:
2
− p3,5 ≥ 2(p1,5 + p2,5 + p4,5 ).
3
(56)
Similary, the constraints for the other negative elements take the form:
−p1,3 ≥ 0
(57)
1
−p4,6 ≥ 3(p1,6 + p2,6 + p3,6 ).
2
(59)
−p2,4 ≥ 0
3.3.1
(58)
Transform from P to ξ
Now each element of P is of one of the following forms:
pi,k ≡ 0
(60)
pi,k ≥ 0
(61)
−pi,k ≥ qk ,
k = i + nα .
(62)
n
ξ
These variables can be transformed and reindexed to a new set {ξl }l=1
such that the inequalities take the
simple form ξl ≥ 0 for each l. This mapping also changes from a double-indexed system (pi,j ) to a single
index (ξl ). The index l is introduced because the zero elements of P are not mapped to ξ, so the mapping
is unique to every matrix. For nξ sets {l, i, k}, each ξl is of one of the following two forms:
ξl = pi,k ,
k 6= i + nα
ξl = −(pi,k + qk ),
k = i + nα .
(63)
(64)
Note that the second set is of size nR and thus the size of the first set is nξ − nR .
For the example, nξ = 12 since there are 12 non-zero elements of P . There are nR = 4 variables whose
transform depends on qk , and thus nξ − nR = 8 variables whose transform does not. The total transform is
given in table 1.
To complete the construction, it remains to specify the probability distribution that governs each variable
ξl . Since ξl ≥ 0, l = 1, . . . , nξ , let
ξl ∼ log N (µξl , ηlξ ).
(65)
The role of the hyperparameters µ and η and how to calibrate them will be explained in detail in the next
section. For ease and generality of notation, let k be the vector of all model parameters (this includes A, b,
E), let ψ be the vector of inadequacy parameters (so far, ξ ∈ ψ and more inadequacy parameters will be
introduced in the upcoming subsections), and let ζ be the vector of all hyperparameters.
This concludes the description of S. Recall that the operator consists of three pieces:
S = Ŝ + A + W.
The next subsections continue with formulations of A and W.
12
(66)
ξi
ξ1
ξ2
ξ3
ξ4
ξ5
ξ6
ξ7
ξ8
ξ9
ξ10
ξ11
ξ12
=
=
=
=
=
=
=
=
=
=
=
=
=
pj,k
p1,1
−p1,3
p1,5
p1,6
p2,2
−p2,4
p2,5
p2,6
−p3,5 − 23 (p1,5 + p3,5 + 23 p4,5 )
p3,6
p4,5
−p4,6 − 3(p1,6 + p2,6 + 12 p3,6 )
Table 1: The transformed variables ξ for the example operator.
3.4
The catchall reactions A
There is much flexibility in the matrix S with respect to how it can redistribute atoms from certain concentrations to others. In fact, it is the most flexible (or general) linear formulation. That is, at every point in
time, a certain species Xi can be redistributed among all other species Xj as long as ρXj ≤ ρXi . Moreover,
the rates at which these processes occur are not set a priori, but are calibrated using the available data. The
random matrix S also provides the flexibility of the catchall species— allowing a place for atoms to go that
might in fact make up a species not included in R but present in D.
However, there is one serious limitation of S due entirely to the linearity: while any species can move to
the catchall species (i.e. H2 O −−→ 2 H0 + O0 ), a catchall species can only directly move to a species made up
of the same type of atom. Therefore, a reaction like the reverse of the previous, namely 2 H0 + O0 −−→ H2 O,
is not allowed. This would require a term that depends on the concentrations of both catchall species, but
in a linear operator this is not possible. On the one hand, in the example reaction with species H2 , O2 , OH,
and H2 O, the catchall species could move back to the reduced set of species since H0 could form H2 and O0
could form O2 . In some cases, this might not be such a serious limitation.
On the other hand, consider a methane reaction that includes the species H2 , O2 , H2 O, CH4 , CO, and
CO2 . Then the operator model species set is H0 , O0 , C0 , H2 , O2 , H2 O, CH4 , CO, and CO2 . Here, S can send
carbon atoms from CH4 , CO, and CO2 into C0 . But then they are stuck: Cn , for any n = 1, 2, . . . , is not
in the reduced set of species. To overcome the linearity limitation, there is a straightforward modification
to the operator: for any species Xi that is made up of more than one type of atom, a non-linear reaction
is included in which the product is Xi and the reactants are the corresponding catchall species. Continuing
with the methane reaction,
κ
1
2 H0 + O0 −→
H2 O
0 κ2
(67)
4 H0 + C −→ CH4
(68)
O + C −→ CO
(69)
2 O + C −→ CO2 .
(70)
0
0 κ3
0
0 κ4
This set of reactions is represented by the non-linear operator A. Note that the form is analogous to a
general reaction model. Thus, the constraints (I) and (II) are automatically satisfied.
This modification introduces nκ reaction rate coefficients κ to be calibrated. Similar to the variables ξ,
each κ is positive, by design. Thus,
κ ∼ log N (µκ , η κ ).
(71)
Then ψ is augmented to include these rate coefficients κ and ζ is augmented to include the additional
13
hyperparameters µκ and η κ .
3.5
The energy operator W
The third and final component of the operator is the non-linear stochastic energy operator W. The role of
W is to account for temperature changes due to atoms moving into and out of the catchall species. In other
words, allowing for the existence of the catchall species endows them with mass; here the catchall formulation
is completed by endowing them with thermodynamic properties. Specifically, this includes internal energy
and specific heat capacity.
Recall the differential equation for dT /dt:

! n
S
X
1
dT

ui (T )ẋi  .
(72)
= W(x, ẋ, T ) = − PnS
dt
i cvi (T )xi
i
For nα < i ≤ nS , cvi (T ) and ui (T ) are known as functions of temperature from the literature on thermodynamic properties of chemical species [34]. The new contribution is to allow for ui (T ) and cvi (T ) for
i = 1, . . . , nα , that is, allow for catchall energies and specific heats and then incorporate these into the calculation of the time derivative of temperature. For actual chemical species, these properties are always given
as a function of temperature. Thus, each new coefficient will also be allowed to have a simple temperaturedependence. Consider a catchall species X0i , i = 1, . . . , nα . For the internal energy, we pose the following
form:
ui (T ) = α0i + α1i T + α2i T 2 ,
(73)
and, since cv is its derivative with respect to temperature,
cvi (T ) = α1i + 2α2i T.
(74)
Then α0 , α1 , and α2 are additional parameters to be calibrated. Furthermore, like all the other random variables introduced during the modeling of the inadequacy operator, each will in fact be represented
by a probability distribution. This is appropriate since we have incorporated some physical information
(temperature-dependence), but the true functional form is uncertain. It is known that α1 and α2 are positive, while α0 could be positive or negative. These properties are exhibited in probability densities of the
form
α
α0 ∼ N (µα
0 , η0 )
α
αl ∼ log N (µα
l , ηl ),
(75)
l = 1, 2.
(76)
Since the above applies to the nα catchall species, there are 3nα new variables. Of course, ψ and ζ are again
augmented to include the new (and final) inadequacy parameters and hyperparameters. Thus, the sets are
the model parameters k = {A, b, E}, the inadequacy parameters ψ = {ξ, κ, α}, and the hyperparameters
ζ = {µξ , η ξ , µκ , η κ , µα , η α }.
This concludes the description of the stochastic operator S. Many model parameters and hyperparameters
have been introduced for the formulation of the reduced and stochastic operator models; the calibration of
these parameters and validation of the models is discussed in § 4.
3.6
Mapping from the operator to typical reaction form
Before presenting the calibration and validation of the models, there is a final detail to consider regarding
the formulation of the operator. A natural question after inspection of the operator is: What does this look
like in terms of typical chemical reactions? The answer connects the action of the operator Ŝ to the physical
14
interpretation of chemical reactions. It is now demonstrated that the random matrix Ŝ = L−1 SL can be
k P
mapped to a typical chemical reaction of the form A −
→
βB.
Theorem 3.2. For every j = 1, . . . , nS , the jth column of Ŝ corresponds to the reaction
kj
Xj −→
where kj = ŝjj and βjp =
ŝjp
|ŝjj |
X
βjp Xp ,
(77)
p6=j
.
Proof. Let x be the vector of concentrations of length n (drop the subscript S for ease of notation). Let
the set of reactions above be denoted L(x) (in the same way that the reduced mechanism model is written
R(x)). We will show Ŝx = L(x), element-wise.
First,


ŝ1,1 x1 + ŝ1,2 x2 + · · · + ŝ1,n xn


 ŝ2,1 x1 + ŝ2,2 x2 + · · · + ŝ2,n xn 

,
(78)
Ŝx = 
..

.


ŝn,1 x1 + ŝn,2 x2 + · · · + ŝn,n xn
and for a single species Xi ,
(Ŝx)i =
X
ŝij xj .
(79)
j
Now consider L(x). The rate for a particular Xi consists of multiple terms: one in which Xi is the chemical
reactant, and n − 1 terms in which Xi is the chemical product. When Xi is a reactant, the corresponding
rate is −ki xi = ŝii xi . When Xi is a product (and Xj is the reactant, j 6= i), the rates from each reaction are
!
ŝij
(80)
+ kj βij xj = ŝjj xj = ŝij xj , j 6= i.
ŝjj
Putting the two terms together, we have
(L(x))i = ŝii xi +
X
ŝij xj
(81)
j6=i
=
X
ŝij xj
(82)
j
= (Ŝx)i .
4
(83)
Calibration and validation
This section describes a Bayesian approach to model calibration and validation for both the original reduced
chemical model and the reduced model with the stochastic inadequacy operator described in § 3. The
original reduced model is calibrated using a typical Bayesian formulation, as described in § 4.1. Because
of the introduction of the hyperparameters in the inadequacy operator, it is most straightforward to use a
hierarchical Bayesian approach for calibration of the inadequacy operator model. This approach is given in
§ 4.2. Finally, techniques from posterior predictive model assessment are used to validate (or invalidate)
both models, as discussed in § 4.3.
15
4.1
Calibration of the reduced model
This section presents the specifics of the calibration of the reduced model as a standard Bayesian inverse
problem (as opposed to the upcoming hierarchical scheme). The parameters one wishes to learn are the
constants in the model for the rate coefficients for each reaction in the reduced set—i.e., the coefficients A,
b, and E introduced in § 2.
The data that is to be used to infer these reaction rate parameters are observations generated by the
detailed chemical kinetics model. Specifically, the calibration data set consists of observations of the molar
concentrations of each of the nR species tracked by the reduced model and temperature, at nt instances
in time, and for nIC initial conditions. The initial condition is given by the set {xf , xo , T }|t=0 and can be
characterized by just two quantities: the equivalence ratio φ and T . The equivalence ratio quantifies how
far the initial condition deviates from the stoichiometric ratio of fuel to oxidizer and is defined by
φ=
xf /xo
,
xfST O /xoST O
(84)
where xfST O and xoST O denote the stoichiometric concentrations of fuel and oxidizer, respectively. Thus, the
initial condition is written as the set IC = {φ, T (t = 0)}.
The observations are collected into the vector d, and it is assumed that the data are contaminated by
additive Gaussian noise, such that
d = {dijl },
where
dijl = dtijl + ijl .
and ∼ N (0, σ2 ). The notation dt indicates the “true” value of the observable, which is supplied by the
detailed model:
dtijl = xD
i (tj , ICl ),
i = 1, . . . , nR ;
dtijl
i = nR + 1.
D
= T (tj , ICl ),
j = 1, . . . , nt ;
l = 1, . . . , nIC ;
(85)
(86)
In the calibration of the reduced model, nIC = 1. It will be shown that the reduced model cannot be
valid even for a single initial condition. Therefore, the reduced model cannot have the flexibility to apply
to multiple scenarios. However, for the calibration of the stochastic operator model, nIC > 1. In that case,
calibrating against multiple initial conditions is done so that the calibrated inadequacy operator can be used
over a range of scenarios, including possible prediction scenarios.
To set up the calibration of R, assume that the reduced model does in fact represent the reaction that
generated the data, and that the only error is in the measurements. (Later, validation will show that this
is in fact incorrect.) This implies that each observed value, dijl , is equal to the model output plus some
measurement error. Therefore, the data model is
dijl = MR
ijl (k) + ijl .
(87)
where MR
ijl denotes the mapping from the calibration parameters k to the observable indexed by i, j, l—i.e.,
the concentration of species i (or temperature if i = nR + 1) at time j and for initial condition l. For
simplicity of exposition in the following, this can be reindexed:
di = MR
i (k) + i ,
i = 1, . . . , nd .
(88)
The model parameters of the reduced model are the parameters for the Arrhenius reaction rate model
for the mR reaction rates, where each k = AT b exp(−E/R◦ T ). The vector k of calibration parameters is
then: k = [A1 , . . . , AmR , b1 , . . . , bmR , E1 , . . . , EmR ]T . The posterior distribution for the model parameters k
16
given the data d is given by Bayes’ Theorem:
p(k|d) ∝ p(d|k)p(k),
(89)
where p(k) is a prior distribution of the parameters representing knowledge of k before considering the data.
The prior for A in each reaction rate is taken to be an independent lognormal distribution since this parameter is known to be positive. For b and E, the prior is chosen with an independent Gaussian distribution.
For all these, the prior has mean µ equal to the nominal value, i.e. the value given for the corresponding
elementary reaction in the detailed model, and a standard deviation that is 10% of the nominal value. These
comprise the complete set of model parameters for the reduced model; the joint prior is a product of these:
p(k) = p(A)p(b)p(E),
(90)
A
Ai ∼ log N (µA
i , ηi )
(91)
where
bi ∼
Ei ∼
N (µbi , ηib )
E
N (µE
i , ηi ),
(92)
(93)
and i = 1, . . . , mR .
The likelihood function p(d|k) represents how plausible it is that the data d arose from the model with
the specific values of the parameters k. With a Gaussian data model and nd data points, the likelihood
takes the form:
1
1
R
T −1
R
(d
−
M
(k))
Σ
(d
−
M
(k))
,
(94)
exp
−
p(d|k) =
2
(2π)nd /2 |Σ|1/2
where Σ is the diagonal matrix of variances corresponding to the measurement error i of the nd observations.
Now that the prior and likelihood are determined, the posterior p(k|d) is defined by (89).
The posterior distribution can be sampled using Markov chain Monte Carlo sampling methods [16, 23, 25].
The algorithm used here to sample the posterior distribution is the Delayed Rejection Adaptive Metroplis
(DRAM) algorithm [25]. This technique is a modification of the Metropolis Hastings algorithm, which is a
type of Markov chain Monte Carlo (MCMC) algorithm. DRAM combines two modifications to Metropolis
Hastings: Delayed Rejection [24, 38] and Adaptive Metropolis [26]. Delayed Rejection (DR) allows for
multiple proposals before rejection. The acceptance probability of each stage is computed to maintain
reversibility of the chain. Adaptive Metropolis (AM) updates the proposal covariance at given intervals
during generation of the chain. Together, DR and AM constitute DRAM.
Finally, to implement the above sampling algorithm, the software QUESO (Quantification of Uncertainty for Estimation, Simulation, and Optimization) is used[13, 43]. QUESO is a statistical numerical
library designed for research on statistical forward and inverse problems, and can be run in multiprocessor
environments. There are other software libraries available to sample posterior distributions including BUGS
(Bayesian Inference Using Gibbs Sampling) [33] and MUQ (MIT Uncertainty Quantification library) [2].
4.2
Calibration of the inadequacy operator model
As shown in § 3, the parameters of S, A, and W are characterized by probability distributions whose
hyperparameters must also be calibrated. Because of this model structure—i.e., that some of the parameters
of interest are actually hyperparameters characterizing the probability density associated with the parameters
that appear directly in the model—it is natural to pose the calibration problem within the hierarchical
Bayesian modeling framework described in the work of Berliner [6, 52].
17
4.2.1
Hierarchical Bayesian modeling
The calibration problem is formulated as a single Bayesian update for the hyperparameters ζ, the inadequacy
model parameters ψ, and the chemistry model parameters k given the observations d. Bayes’ theorem
requires that
p(ψ, ζ, k|d) ∝ p(d|ψ, ζ, k) p(ψ, ζ, k).
This form can be simplified using the hierarchical structure of the model. Specifically, because the model
only depends on ζ indirectly through ψ, the values of ζ are irrelevant after conditioning on ψ. Thus, the
likelihood becomes
p(d|ψ, ζ, k) = p(d|ψ, k).
The prior can be expanded as follows:
p(ψ, ζ, k) = p(ψ|ζ, k) p(ζ, k) = p(ψ|ζ) p(ζ, k),
because the inadequacy model parameters ψ are independent of k in the prior. Further assuming that ζ
and k are independent in the prior, the posterior distribution can be written as
p(ψ, ζ, k|d) ∝ p(d|ψ, k) p(ψ|ζ) p(ζ) p(k).
Clearly, the posterior represents the joint distribution for model parameters k, the hyperparameters ζ, and
the inadequacy parameters ψ conditioned on the data. However, the particular values of the inadequacy
parameters ψ that are preferred by the given data are not necessarily of interest here because the goal is for
the formulation to be applicable to a broad range of problems, including scenarios outside the calibration data
set. In this situation, the hyperparameters ζ are the primary target of the inference rather than ψ, and one
can marginalize over ψ to find the joint distribution for the model parameters k with the hyperparameters:
Z
p(ζ, k|d) = p(ψ, ζ, k|d)dψ.
(95)
This joint posterior is equivalent to that found by formulating the following inverse problem:
p(ζ, k|d) = p(d|ζ, k) p(ζ) p(k),
where the likelihood is given by
Z
p(d|ζ, k) =
4.2.2
p(d|ζ, ψ, k)p(ψ|ζ)dψ.
Inverse problem details
The data set is similar to the previous case, but nIC > 1. Instead of the parameter-to-observable map for
the reduced model MR , which is invalidated in the upcoming examples, now consider MS . Specifically, MS
is the mapping from the rate constants k and model inadequacy parameters ψ to the observables induced by
reduced chemistry model after being enriched by the stochastic operator inadequacy representation. Further,
the observation error is the same as in § 4.1, and the dataset may be reindexed in the same way such that
di = MSi (ψ, k)) + i ,
i = 1, . . . , nd .
(96)
Thus, taking into account the Gaussian measurement error, the likelihood is given by
p(d|ψ, ζ, k) = p(d|ψ, k)
=
1
(2π)nd /2 |Σ|1/2
(97)
1
exp − (d − MS (ψ, k))T Σ−1 (d − MS (ψ, k))
2
18
(98)
where nd = (nR + 1) × nt × nIC (number of species, times of measurement, initial conditions) is the total
number of observations.
The prior for the model parameters p(k) is the same here as in § 4.1. Following the hierarchical scheme,
a conditional prior distribution p(ψ|ζ) for the inadequacy parameters given the hyperparameters is also
required. This is inherited from the proposed structure in section 3, given in lines (65), (71), (75), (76).
That is, the conditional prior distribution of each inadequacy parameter is the following:
ξi ∼ log N (µξi , ηiξ ),
κi ∼
α0i ∼
αli ∼
log N (µκi , ηiκ ),
α
N (µα
i
0i , η0i ),
α
α
log N (µli , ηli ),
i = 1, . . . , nξ
(99)
i = 1, . . . , nκ
(100)
= 1, . . . , nα
l = 1, 2, i = 1, . . . , 2nα .
(101)
(102)
Recall ξ are the inadequacy parameters of S, κ are those of A, and α of W. The following prior distributions
are used for the hyperparameters:
(·)
(·)
(·)
µi ∼ N (µµi , ηiµ )
(103)
∼ J (0, ∞),
(104)
(·)
ηi
where (·) represents ξ, κ, or α. In one dimension x, the Jeffreys distribution pJ (x) ∼ J (0, ∞) is given by
pJ (x) =
1
,
x
x ∈ (0, ∞).
(105)
This cannot be normalized (it is an improper distribution), but it can still be used as a prior distribution
[18].
As described previously, the posterior distribution p(ψ, ζ, k|d) is sampled using Delayed Rejection Adaptive Metroplis (DRAM) [25].
4.3
Validation
Once a model has been constructed and calibrated, the next step is validation; that is, the process of checking
if data obtained from observations of the modeled system are consistent with the calibrated model, given
uncertainties in the model parameters, the model inadequacy and the observational errors. The validation
approach used here is that of posterior predictive assessment [22, 44].
v
Consider a set of observations of the system {vi }ni=1
. This set will in general include the data d used
for calibration and may also include additional observations of the same or different quantities not used in
calibration. However, there is an observational error for each observation so that the observed value vi is
related to the unknown true value vit by
vi = vit + i .
(106)
The calibrated model makes a claim about the distribution of plausible values of vit given by p(vit |d). The
relevant validation question is whether the observations vi are consistent with the model’s claim regarding
the observation. This is given by
Z
Z
t
t
t
p(vi |d) =
p(vi |vi )p(vi |d) dvi =
p (vi − vit )p(vit |d) dvit
(107)
dt
dt
where p is the probability distribution of the observation errors.
Finally, the posterior distribution of vit is determined from the calibrated distributions of the model
parameters θ, yielding
Z
Z
t
t
p(vi |d) =
p (vi − vi )
p(vi |θ)p(θ|d) dθ dvit .
(108)
dt
θ
19
Here, depending on the circumstance, the parameters θ can include the physical parameters k, the hyperparameters ζ and/or the inadequacy distributions ψ.
In the case of the chemical kinetics models considered here, three different validation situations are relevant. First, in testing the reduced model itself (without inadequacy), θ includes only the kinetic parameters
k. Second, when testing the form of the inadequacy model to determine whether it is sufficient to represent
the observed discrepancy with the calibration data, the question is whether the inadequacy form, considered
to be deterministic in ψ, can correct the model relative to the calibration data. In this case, θ includes ψ
and k and the posterior of vit is given by
Z Z
t
p(vi |d) =
p(vit |k, ψ)p(k, ψ|d) dψ dk.
(109)
k
ψ
In the third situation, one tests whether the stochastic inadequacy representation can account for model
discrepancies over a broad range of conditions, particularly for conditions not included in the calibration.
Here, the inadequacy form is stochastic in ψ and this stochasticity is characterized by the hyperparameters
ζ. In this case, the posterior of vit is
Z Z Z
t
p(vi |d) =
p(vit |k, ψ, ζ)p(ψ|ζ)p(k, ζ|d)dζdψdk.
(110)
k
ψ
ζ
The integral (108) yields the posterior prediction of the observation vi which can be used to find the
total probability of observing a value less probable than the actual observation. As explained in [42], this
probability can be used as a validation metric, which in turn makes use of highest probability density (HPD)
credibility regions [8]. In [42], the β-HPD (0 ≤ β ≤ 1) credibility region S is the set for which the probability
of belonging to S is β and the probability density for any point inside S is higher than those outside. Define
for one observation vi ,
γi = 1 − βmini ,
(111)
where βmini is the smallest value of β for which vi ∈ Si . Another way to think of γi is that it is the integral
of p(vit |d) over the domain Vi = {vit : p(vit |d) < p(vi |d)}. For samples {vij }Jj=1 of this distribution p(vit |d),
we have
Z
p(vit |d)dvit
(112)
γi =
Vi
1X
1vij ∈Vi .
'
J j
(113)
A delicate point here is the choice of tolerance τ : if γ < τ , the model has been shown to be inconsistent
with the observation(s). A typical value for the tolerance is 0.05, although there is an extensive discussion in
the statistics literature about how to interpret this [21, 37, 42]. When comparing multiple observations but
treating them as independent, as we will later on, the tolerance should be corrected and set lower because
with many observations of a random variable it is more likely to make a low-probability observation. The
Bonferroni correction suggests dividing the tolerance by the number of points [7]. Ideally, all data points will
be clearly consistent with the model output (the model is not invalidated), or, at least one will be clearly
inconsistent (the model is invalid and thus inadequate).
5
Hydrogen combustion
As an example, the proposed inadequacy operator for a chemical mechanism model of hydrogen combustion
is developed. Since there are several sources of information to consider while implementing the proposed
approach, it is helpful to summarize the steps as such:
1. Identify detailed and reduced kinetics models.
20
2. Calibrate the reduced model using Bayesian inference and data from the detailed model.
3. Use a posterior predictive check to validate the reduced model.
4. Represent the inadequacy (if invalid).
5. Recalibrate the reduced model along with the inadequacy model.
6. Use a posterior predictive check to validate the new model.
7. Make a prediction (if not invalidated).
5.1
Identification of the detailed and reduced models.
Both the detailed model and its reduced version are described in [54]. In the detailed model, there are two
types of atoms: hydrogen and oxygen; eight distinct species: H2 , O2 , H, O, OH, HO2 , H2 O, H2 O2 ; and
twenty-one elementary reactions. The reduced model includes five of the given twenty-one reactions, and
there are seven species tracked instead of eight. The resulting differential equations are much simpler than
those given by the full model. Both the twenty-one- and five-step reaction mechanisms and corresponding
forward reaction rates are listed in appendix A.
5.2
Calibration and validation of the reduced model
The first step is to calibrate the coefficients that make up the five reaction rates k using data generated by
the detailed model. The observations are taken of each of the seven species tracked by the reduced model
plus temperature, at five instances in time, and for one initial condition:
d = {dijl }, where dijl = xD
i (tj , ICl ),
i = 1, . . . , 7;
j = 1, . . . , 5;
l = 1,
(114)
and
dijl = TiD (tj , ICl ),
i = 8;
j = 1, . . . , 5;
l = 1.
(115)
Reindexing,
d
d = {di }ni=1
,
(116)
where nd = 8 × 5 × 1 = 40. The set of time points is {20, 40, 60, 80, 100}µs. The initial condition is φ = 1.0,
T0 = 1500 K. As usual, it is assumed that each observed value, di , is equal to the model output MR
i (k) plus
some measurement error, i ∼ N (0, σ2i ). Thus, the data model is:
di = MR
i (k) + i .
(117)
As explained in § 4.1, the prior is p(k) = p(A)p(b)p(E), where
A
Ai ∼ log N (µA
i , ηi )
bi ∼
Ei ∼
N (µbi , ηib )
E
N (µE
i , ηi ),
and i = 1, . . . , 5. Given the data model above, the likelihood is:
1
1
R
T −1
R
(d
−
M
(k))
Σ
(d
−
M
(k))
.
exp
−
p(d|k) =
2
(2π)40/2 |Σ|1/2
(118)
(119)
(120)
(121)
After calibrating the model parameters k, the model output is compared to the observations. Figure 2
shows the observations, generated by the detailed model D, compared to the reduced model R output. The
distributions of the model account for parametric and measurement uncertainty. It is clear that some of the
21
observations are not a plausible outcome of the reduced model, even with the calibrated parameters. There
is also a severe difference between temperatures, shown in figure 3. Note that in each figure, the model
output is shown with the mean and error bars corresponding to one and two standard deviations. The
probability that many of the observations came from a system described by the reduced model is extremely
low, especially those of H2 , H, H2 O, and temperature. By inspection of figures 2-3, it is obvious that several
γ-values are essentially zero: the reduced model is thus deemed invalid and inadequate.
To account for the model inadequacy demonstrated above, the inadequacy operator S will now be constructed.
5.3
Formulation of the inadequacy operator S
First we have,
xS = [x1 , x2 , . . . , x9 ]T ,
(122)
with the concentrations given in the order of H0 , O0 , H2 , O2 , H, O, OH, HO2 , H2 O. Note mR = 5, nR = 7,
nα = 2, nS = 9.
5.3.1
The random matrix S
From theorem 3.1, we know that the matrix has the

s1,1
0
s1,3
0
 0
s
0
s
2,2
2,4

s3,1
0
s3,3
0

 0
s
0
s
4,2
4,4

0
s5,3
0
S=
s5,1
 0
s
0
s
6,2
6,4

 0
0
0
0

 0
0
0
0
0
0
0
0
following structure:
s1,5
0
s3,5
0
s5,5
0
0
0
0
0
s2,6
0
s4,6
0
s6,6
0
0
0
s1,7
s2,7
s3,7
s4,7
s5,7
s6,7
s7,7
s8,7
s9,7
s1,8
s2,8
s3,8
s4,8
s5,8
s6,8
s7,8
s8,8
s9,8

s1,9
s2,9 

s3,9 

s4,9 

s5,9 
.
s6,9 

s7,9 

s8,9 
s9,9
Here, of the 81 entries of S, 42 are identically zero. Next E is the nα × nS matrix:
1 0 1 0 1 0 1/2 1/3 2/3
E=
.
0 1 0 1 0 1 1/2 2/3 1/3
The first row of E corresponds to hydrogen
matrix whose columns span null(E):

−1 0
 0 −1

1
0

0
1

0
0
C=

0
0

0
0

0
0
0
0
(123)
(124)
atoms and the second to oxygen. C is the following nS × nR
−1
0
0
0
1
0
0
0
0
0
−1
0
0
0
1
0
0
0
22
−1/2 −1/3
−1/2 −2/3
0
0
0
0
0
0
0
0
1
0
0
1
0
0

−2/3
−1/3

0 

0 

0 
.
0 

0 

0 
1
(125)
0.30
0.50
0.25
0.45
0.20
O2
H2
0.55
0.40
0.15
0.35
0.10
0.30
0.00002
0.00004
0.00006
Time [s]
0.00008
0.05
0.00002
0.00010
0.8
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.6
0.4
O
H
0.2
0.1
0.2
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
0.0
0.00002
0.00010
0.30
1.6
0.25
H2 O
OH
1.4
0.20
1.2
0.15
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.0
0.00002
0.00010
Figure 2: Concentrations [mol/m3 ] versus time [s] for H2 /O2 reaction, φ = 1.0, T0 = 1500 K. Observations
(red), reduced model R (blue).
23
3000
Temperature
2800
2600
2400
2200
2000
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
Figure 3: Temperature [K] versus time [s] for H2 /O2 reaction, φ = 1.0, T0 = 1500 K. Observations (red),
reduced model R (blue).
P is an nR × nS matrix, where S = CP :

p1,1
0
p1,3
 0
p
0
2,2

p3,1
0
p
3,3

p4,2
0
P =
 0
 0
0
0

 0
0
0
0
0
0
0
p2,4
0
p4,4
0
0
0
p1,5
0
p3,5
0
0
0
0
0
p2,6
0
p4,6
0
0
0
p1,7
p2,7
p3,7
p4,7
p5,7
p6,7
p7,7
p1,8
p2,8
p3,8
p4,8
p5,8
p6,8
p7,8

p1,9
p2,9 

p3,9 

p4,9 
.
p5,9 

p6,9 
p7,9
(126)
The transform is given in table 2, and the constraints are now
ξi ≥ 0,
i = 1, . . . , 33.
(127)
Finally, to complete the formulation of S,
ξi ∼ log N (µξi , ηiξ ),
5.3.2
i = 1, . . . , 33.
(128)
The catchall reactions A
The catchall reactions allow the catchall species to directly form any species made up of more than one type
of atom. Otherwise, that reaction is already allowed via S (H0 −−→ H is allowed by S for example). Thus,
there are three catchall reactions:
κ
1
H0 + O0 −→
OH
(129)
H + 2 O −→ HO2
(130)
0
0 κ2
0 κ3
2 H0 + O −→ H2 O.
(131)
The reaction rate coefficients are denoted κ, and these are included in the set of inadequacy parameters.
Like the variables ξ, each κ will be modeled with a lognormal distribution whose hyperparameters are also
24
ξi
ξ1
ξ2
ξ3
ξ4
ξ5
ξ6
ξ7
ξ8
ξ9
ξ10
ξ11
ξ12
ξ13
ξ14
ξ15
ξ16
ξ17
ξ18
ξ19
ξ20
ξ21
ξ22
ξ23
ξ24
ξ25
ξ26
ξ27
ξ28
ξ29
ξ30
ξ31
ξ32
ξ33
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
pj,k
p1,1
−(p1,3 + p3,3 )
p1,5
p1,7
p1,8
p1,9
p2,2
−(p2,4 + p4,4 )
p2,6
p2,7
p2,8
p2,9
p3,1
p3,3
−(p3,5 + p1,5 )
p3,7
p3,8
p3,9
p4,2
p4,4
−(p4,6 + p2,6 )
p4,7
p4,8
p4,9
−p5,7 − 2(p1,7 + p2,7 + p3,7 + p4,7 + (2/3)p6,7 + (2/3)p7,7 )
p5,8
p5,9
p6,7
−p6,8 − 3(p1,8 + p2,8 + p3,8 + p4,8 + (1/2)p5,8 + (2/3)p7,8 )
p6,9
p7,7
p7,8
−p7,9 − 3(p1,9 + p2,9 + p3,9 + p4,9 + (1/2)p5,9 + (2/3)p6,9 )
Table 2: The transformed variables ξ for the H2 /O2 operator.
25
calibrated. From the reactions above, the associated rate of each is:
r10 = κ1 x1 x2
(132)
r20
r30
= κ2 x1 x2
(133)
= κ3 x1 x2 ,
(134)
and the resulting additions to the differential equations for H0 , O0 , OH, HO2 , and H2 O are
H0 :
− r10 − r20 − 2r30
(135)
OH:
+
(137)
HO2 :
+
H2 O:
+
0
O:
−
r10 −
r10
r20
r30 .
2r20
−
r30
(136)
(138)
(139)
The terms above (135) - (139) are written as A(x).
5.3.3
The energy operator W
The third and final piece of the operator S is the energy operator W. Recall

! n
S
X
dT
1

= W(x, ẋ, T ) = − PnS
ui (T )ẋi 
dt
i cvi (T )xi
i
(140)
and so a description of u(T ) and cv (T ) for each catchall species is necessary. To do so, the new parameters
α0 , α1 , and α2 are introduced. That is,
ui (T ) = α0i + α1i T + α2i T 2
(141)
cvi (T ) = α1i + 2α2i T,
(142)
where i = 1 corresponds to H0 and i = 2 to O0 .
5.4
Calibration and validation of the inadequacy operator
There are nine initial conditions given by the combinations of φ = {.9, 1.0, 1.1} and initial temperature
T0 = {1450, 1500, 1550}K. The set of time points is again {20, 40, 60, 80, 100}µs. The prior, likelihood, and
posterior distributions exactly follow from the general form in § 4.2. After calibration, the stochastic operator
with catchall species displays excellent agreement with the data. Table 3 shows the value of gamma for each
data point. Figures 4-12 show the model output of concentrations for the nine different initial conditions.
For some species, such as O2 and O, the concentrations approach zero and the model output appears to be
negative on the edge of its distribution. Note that the model output including the parametric uncertainty
is never negative, as guaranteed by the imposed constraints. However, the measurement error is additive
with mean zero. Thus, accounting for this possible error in the distribution, as plotted, may extend the
distribution past zero. Finally, the temperature output also shows good agreement with the data, shown in
figures 13 and 14.
5.5
Prediction
We conclude this section by predicting the concentrations and temperature at extrapolative conditions. That
is, the initial condition is given by φ = 1.15 and T0 = 1350 K. The prediction is shown for a higher range in
time, i.e. t ∈ [3e − 5, 1.3e − 4]. Although the corresponding output from the detailed model was not used to
calibrate the model, the detailed model output is shown analogously to the previous results.
26
φ
TIC
0.9
1450
0.9
1500
0.9
1550
1.0
1450
1.0
1500
1.0
1550
1.1
1450
1.1
1500
1.1
1550
t (tj )
H2
O2
H
O
OH
HO2
H2 O
T
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
0.64
0.98
0.96
0.52
0.32
0.69
0.81
0.85
0.39
0.2
0.89
0.77
0.72
0.27
0.13
0.78
0.69
0.89
0.44
0.12
0.81
0.56
0.93
0.28
0.06
0.98
0.54
0.76
0.17
0.03
0.93
0.48
0.76
0.52
0.13
0.91
0.41
0.88
0.35
0.07
0.94
0.43
0.93
0.23
0.04
0.25
0.88
0.94
0.9
0.78
0.72
0.8
0.94
0.82
0.7
0.98
0.75
0.98
0.75
0.65
0.36
0.68
0.84
0.87
0.68
0.92
0.61
0.91
0.79
0.58
0.79
0.63
0.96
0.7
0.52
0.54
0.53
0.74
0.9
0.72
0.91
0.51
0.79
0.83
0.62
0.62
0.5
0.9
0.76
0.54
.00005
0.65
0.9
0.61
0.36
0.05
0.43
0.96
0.45
0.26
0.78
0.37
0.83
0.32
0.18
0.0002
0.2
0.69
0.51
0.17
0.39
0.12
0.88
0.33
0.1
0.4
0.12
1.0
0.2
0.05
0.002
0.03
0.36
0.59
0.15
0.73
0.02
0.54
0.39
0.07
0.04
0.02
0.81
0.21
0.03
0.11
0.96
0.98
0.79
0.63
0.4
0.91
0.93
0.69
0.56
0.74
0.84
0.85
0.6
0.47
0.2
0.73
0.93
0.79
0.62
0.64
0.67
0.98
0.68
0.5
0.92
0.67
0.89
0.59
0.42
0.36
0.55
0.81
0.87
0.64
0.92
0.51
0.93
0.75
0.58
0.7
0.53
1.0
0.66
0.48
0.47
0.67
0.75
0.66
0.6
0.91
0.66
0.68
0.57
0.49
0.68
0.65
0.6
0.47
0.41
0.62
0.74
0.78
0.61
0.44
0.8
0.76
0.71
0.49
0.34
0.61
0.74
0.62
0.38
0.26
0.79
0.89
0.98
0.69
0.45
0.73
0.94
0.86
0.57
0.34
0.61
0.93
0.73
0.44
0.25
0.96
0.98
1.0
0.98
0.98
0.96
1.0
0.98
0.96
0.96
0.98
0.96
0.98
0.96
0.94
0.98
0.96
1.0
0.96
1.0
0.96
1.0
0.98
0.98
0.96
0.98
0.95
1.0
0.96
1.0
1.0
0.96
0.98
0.98
0.95
0.96
0.98
1.0
0.98
0.96
1.0
0.98
0.98
0.96
0.98
0.01
0.29
0.22
0.58
0.24
0.33
0.14
0.29
0.73
0.36
0.94
0.11
0.42
0.95
0.77
0.05
0.06
0.07
0.43
1.0
0.78
0.03
0.13
0.75
0.64
0.47
0.03
0.26
0.85
0.36
0.23
0.01
0.03
0.26
0.82
0.67
0.01
0.07
0.56
0.75
0.15
0.01
0.17
0.9
0.41
0.002
0.21
0.49
0.75
0.23
0.53
0.1
0.47
0.69
0.2
0.48
0.07
0.49
0.62
0.18
0.02
0.07
0.32
0.88
0.26
0.96
0.03
0.34
0.74
0.23
0.17
0.03
0.42
0.67
0.2
0.11
0.02
0.23
0.95
0.32
0.44
0.01
0.27
0.84
0.28
0.05
0.01
0.34
0.77
0.25
Table 3: The values of γ for each data point using the H2 /O2 operator.
27
0.4
0.4
0.3
O2
H2
0.5
0.3
0.2
0.1
0.00004
0.00006
Time [s]
0.00008
0.0
0.00002
0.00010
0.8
0.3
0.6
0.2
0.4
0.1
O
H
0.1
0.00002
0.2
0.2
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.0
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.5
1.4
0.25
H2 O
OH
1.3
0.20
1.2
0.15
1.1
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.0
0.00002
0.00010
Figure 4: Concentrations [mol/m3 ] versus time [s], φ = 0.9, T0 = 1450 K. Observations (red), operator
model O (blue).
28
0.4
0.4
0.3
O2
H2
0.5
0.3
0.2
0.1
0.00002
0.2
0.1
0.00004
0.00006
Time [s]
0.00008
0.0
0.00002
0.00010
0.5
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.4
0.2
0.1
O
H
0.3
0.2
0.0
0.1
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.5
1.4
0.25
H2 O
OH
1.3
0.20
1.2
0.15
1.1
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.0
0.00002
0.00010
Figure 5: Concentrations [mol/m3 ] versus time [s] for φ = 0.9, T0 = 1500 K. Observations (red), operator
model O (blue).
29
0.5
0.30
0.25
0.4
O2
H2
0.20
0.3
0.15
0.2
0.10
0.1
0.00002
0.00004
0.00006
Time [s]
0.00008
0.05
0.00002
0.00010
0.5
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.4
0.2
0.1
O
H
0.3
0.2
0.0
0.1
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.5
1.4
0.25
H2 O
OH
1.3
0.20
1.2
0.15
1.1
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.0
0.00002
0.00010
Figure 6: Concentrations [mol/m3 ] versus time [s] for φ = 0.9, T0 = 1550 K. Observations (red), operator
model O (blue).
30
0.3
0.4
0.2
O2
H2
0.5
0.3
0.00004
0.00006
Time [s]
0.00008
0.0
0.00002
0.00010
0.8
0.3
0.6
0.2
0.4
0.1
O
H
0.2
0.00002
0.1
0.2
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.0
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.6
1.5
0.25
H2 O
OH
1.4
0.20
1.3
0.15
1.2
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.1
0.00002
0.00010
Figure 7: Concentrations [mol/m3 ] versus time [s] for φ = 1.0, T0 = 1450 K. Observations (red), operator
model O (blue).
31
0.3
0.4
0.2
O2
H2
0.5
0.3
0.2
0.00002
0.1
0.00004
0.00006
Time [s]
0.00008
0.0
0.00002
0.00010
0.6
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.2
0.1
O
H
0.4
0.2
0.0
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.6
1.5
0.25
H2 O
OH
1.4
0.20
1.3
0.15
1.2
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.1
0.00002
0.00010
Figure 8: Concentrations [mol/m3 ] versus time [s] for φ = 1.0, T0 = 1500 K. Observations (red), operator
model O (blue).
32
0.3
0.4
0.2
O2
H2
0.5
0.3
0.2
0.00002
0.1
0.00004
0.00006
Time [s]
0.00008
0.0
0.00002
0.00010
0.6
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.2
0.1
O
H
0.4
0.2
0.0
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.6
1.5
0.25
H2 O
OH
1.4
0.20
1.3
0.15
1.2
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.1
0.00002
0.00010
Figure 9: Concentrations [mol/m3 ] versus time [s] for φ = 1.0, T0 = 1550 K. Observations (red), operator
model O (blue).
33
0.6
0.3
0.2
O2
H2
0.5
0.1
0.4
0.0
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.8
0.3
0.6
0.2
0.4
0.1
O
H
0.3
0.00002
0.2
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.0
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.7
1.6
0.25
H2 O
OH
1.5
0.20
1.4
0.15
1.3
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.2
0.00002
0.00010
Figure 10: Concentrations [mol/m3 ] versus time [s] for φ = 1.1, T0 = 1450 K. Observations (red), operator
model O (blue).
34
0.6
0.3
0.2
O2
H2
0.5
0.1
0.4
0.0
0.3
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.6
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.2
0.1
O
H
0.4
0.2
0.0
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.7
1.6
0.25
H2 O
OH
1.5
0.20
1.4
0.15
1.3
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.2
0.00002
0.00010
Figure 11: Concentrations [mol/m3 ] versus time [s] for φ = 1.1, T0 = 1500 K. Observations (red), operator
model O (blue).
35
0.6
0.3
0.2
O2
H2
0.5
0.1
0.4
0.0
0.3
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.6
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.00004
0.00006
Time [s]
0.00008
0.00010
0.3
0.2
0.1
O
H
0.4
0.2
0.0
0.0
0.00002
0.00004
0.00006
Time [s]
0.00008
−0.1
0.00002
0.00010
0.30
1.7
1.6
0.25
H2 O
OH
1.5
0.20
1.4
0.15
1.3
0.10
0.00002
0.00004
0.00006
Time [s]
0.00008
1.2
0.00002
0.00010
Figure 12: Concentrations [mol/m3 ] versus time [s] for φ = 1.1, T0 = 1550 K. Observations (red), operator
model O (blue).
36
3500
3500
3000
Temperature
Temperature
3000
2500
2500
2000
1500
0.00002
0.00004
0.00006
Time [s]
0.00008
2000
0.00002
0.00010
(a) φ = 0.9, T0 = 1450 K
0.00004
0.00006
Time [s]
0.00008
0.00010
(b) φ = 1.0, T0 = 1450 K
3000
3000
Temperature
3500
Temperature
3500
2500
2000
0.00002
2500
0.00004
0.00006
Time [s]
0.00008
2000
0.00002
0.00010
(c) φ = 0.9, T0 = 1500 K
0.00004
0.00006
Time [s]
0.00008
0.00010
(d) φ = 1.0, T0 = 1500 K
3000
3000
Temperature
3500
Temperature
3500
2500
2000
0.00002
2500
0.00004
0.00006
Time [s]
0.00008
2000
0.00002
0.00010
(e) φ = 0.9, T0 = 1550 K
0.00004
0.00006
Time [s]
0.00008
0.00010
(f) φ = 1.0, T0 = 1550 K
Figure 13: Temperature [K] versus time [s] for H2 /O2 reaction, φ = {0.9, 1.0}, T0 = {1450, 1500, 1550}K.
Observations (red), operator model O (blue).
37
3500
Temperature
3000
2500
2000
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
(a) φ = 1.1, T0 = 1450 K
3500
Temperature
3000
2500
2000
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
(b) φ = 1.1, T0 = 1500 K
3500
Temperature
3000
2500
2000
0.00002
0.00004
0.00006
Time [s]
0.00008
0.00010
(c) φ = 1.1, T0 = 1550 K
Figure 14: Temperature [K] versus time [s] for H2 /O2 reaction, φ = 1.1, T0 = {1450, 1500, 1550}K. Observations (red), operator model O (blue).
38
0.7
0.20
0.15
0.6
O2
H2
0.10
0.5
0.05
0.4
0.00
0.3
0.00004
0.00006
0.00008
Time [s]
0.00010
−0.05
0.00012
0.00004
0.00006
0.00008
Time [s]
0.00010
0.00012
0.00004
0.00006
0.00008
Time [s]
0.00010
0.00012
0.00004
0.00006
0.00008
Time [s]
0.00010
0.00012
0.20
0.6
0.15
0.10
O
H
0.4
0.05
0.2
0.00
0.0
0.00004
0.00006
0.00008
Time [s]
0.00010
−0.05
0.00012
1.8
0.25
1.7
0.20
1.6
OH
H2 O
0.30
0.15
1.5
0.10
1.4
0.05
0.00004
0.00006
0.00008
Time [s]
0.00010
1.3
0.00012
Figure 15: Predicted concentrations [mol/m3 ] versus time [s] for φ = 1.15, T0 = 1350 K. Observations (red),
operator model O (blue).
39
3500
Temperature
3000
2500
2000
0.00004
0.00006
0.00008
Time [s]
0.00010
0.00012
Figure 16: Predicted temperature [K] versus time [s] for φ = 1.15, T0 = 1350 K. Observations (red), operator
model O (blue).
φ
1.15
TIC
j (tj )
H2
O2
H
O
OH
HO2
H2 O
T
1350
1
2
3
4
5
0.75
0.68
0.87
0.28
0.06
0.27
0.72
0.92
0.83
0.73
0.0
0.23
0.96
0.31
0.09
0.12
0.81
0.98
0.78
0.66
0.85
0.91
0.82
0.60
0.49
0.99
0.98
0.98
0.99
0.96
0.01
0.02
0.07
0.36
0.73
0.003
0.16
0.79
0.35
0.07
Table 4: The values of γ for the prediction case.
As shown in figures 15 and 16, all observations are consistent with the prediction except the first time
point of H. This suggests that the operator model may benefit from a richer set of calibration data for small
times. The γ-values are shown in table 4.
6
Conclusion
This study addresses the critical problem of model inadequacy that affects nearly all mathematical models of
physical systems. A new approach is developed that combines the flexibility and generality of a probabilistic
model with the available deterministic physical information. In the context of predictive models, these two
properties are essential: flexibility allows the model to be extended to new scenarios of interest, and respecting physical constraints ensures that the predictions will still be physically meaningful. This inadequacy
representation is formulated as a stochastic operator, the bulk of which is described by a random matrix.
The stochastic operator S was developed to account for the inadequacy of a reduced chemical mechanism
and it contains three main components: 1) the random matrix S, 2) the nonlinear catchall reactions A,
and 3) the energy operator W. The random matrix S contains most of the information in S and has some
interesting properties. Typically, the matrix has many identically zero entries. It always has a negative
diagonal, is diagonally dominant, and has non-positive eigenvalues. The reactions in A allow any species in
the reduced model to be the chemical product of the corresponding catchall species (if this is not already
possible through S). Both S and A guarantee conservation of atoms and non-negativity of concentrations.
Finally, the energy operator W modifies the time derivative of temperature by endowing the catchall species
with thermodynamic properties.
The inadequacy operator is tested on an example H2 /O2 mechanism. All observations are plausible
outcomes of the operator model: the concentrations and temperature values from the operator model O
40
are consistent with the data supplied by the detailed model D. Moreover, the prediction showed that the
operator model output was consistent with the (unused) data at all points except one.
There are many avenues for extending the work reported here. Ongoing research includes the application
of the operator to a more complex chemical setting, namely, a methane-air mechanism. We are also investigating a more complex physical setting with a true prediction problem: the prediction of a hydrogen laminar
flame using the calibrated operator found in this work. Other goals are including more physical information
into the inadequacy representation such as realistic temperature-dependence, and using stronger priors based
on knowledge of the chemical reactions and the physical setup. This leads to the next major opportunity for
future work: developing the connection between the stochastic operator and the actual chemistry. Mapping
between the random matrix and the typical chemical reactions was a first step in this direction. However, a
better understanding of what the stochastic operator means in physical terms is needed. This includes not
just the structure, but also the uncertainty in the calibrated parameters. A future goal is to infer something
about the missing chemistry from the calibrated operator. This is important when developing mechanisms
based on experimental data, when no detailed mechanism is available.
Another area for future work is to develop variations of the operator. For example, instead of using the
random matrix S and the catchall reactions in A, a simplified version is the following: given nR species in the
reduced model, include nR reversible dissociation reactions where the reactant is one of the original species
and the products are the corresponding catchall species. In the reverse combination reaction, the catchall
atoms react to form any of the original species. In this fashion, the atoms of any species could move to any
other species in two steps. This representation would lose information held in the current formulation; on
the other hand, it would decrease the number of random variables and thus would be more tractable in more
complex reactions. With a smaller number of additional reactions, the corresponding reaction rates could
then be enriched with temperature-dependence.
Another variation could be a more complete set of nonlinear reactions. Instead of only allowing the
nonlinear catchall reactions, one could augment the reduced model with all or some subset of all possible
nonlinear terms. In contrast to the first variation, this would increase the number of random variables. A
formulation like this might only be possible with more informative priors or knowledge about the chemical
system.
Finally, it would be very interesting to apply this method to new problems. First, the inadequacy operator
could be tested by a more realistic combustion problem. It may be that doing so requires a more complete
thermodynamic description of the catchall species. Another idea is to use the guiding principles of this work
(respecting physical constraints, maintaining flexibility, starting with a linearized version) and developing an
analogous operator (possibly random matrix) in a different physical context. Crossing into another domain
could bring to light many new challenges and common strengths for the stochastic operator approach to
representing model inadequacy.
41
Reaction
Hydrogen-oxygen chain
1. H + O2 −−→ OH + O
2. H2 + O −−→ OH + H
3. H2 + OH −−→ H2 O + H
4. H2 O + O −−→ OH + OH
A
b
E
3.52 × 1016
5.06 × 104
1.17 × 109
7.60 × 100
-0.7
2.7
1.3
3.8
71.4
26.3
15.2
53.4
Direct recombination
5. H + H + M −−→ H2 + M
6. H + OH + M −−→ H2 O + M
7. O + O + M −−→ O2 + M
8. H + O + M −−→ OH + M
9. O + OH + M −−→ HO2 + M
1.30 × 1018
4.00 × 1022
6.17 × 1015
4.71 × 1018
8.00 × 1015
-1.0
-2.0
-0.5
-1.0
0.0
0.0
0.0
0.0
0.0
0.0
Hydroperoxyl reactions
10. H + O2 + M −−→ HO2 + M
11. HO2 + H −−→ OH + OH
12. HO2 + H −−→ H2 + O2
13. HO2 + H −−→ H2 O + O
14. HO2 + O −−→ OH + O2
15. HO2 + OH −−→ H2 O + O2
5.75 × 1019
7.08 × 1013
1.66 × 1013
3.10 × 1013
2.00 × 1013
2.89 × 1013
-1.4
0.0
0.0
0.0
0.0
0.0
0.0
1.2
3.4
7.2
0.0
-2.1
Hydrogen peroxide reactions
16. OH + OH + M −−→ H2 O2 + M
17. HO2 + HO2 −−→ H2 O2 + O2
18. H2 O2 + H −−→ HO2 + H2
19. H2 O2 + H −−→ H2 O + OH
20. H2 O2 + OH −−→ H2 O + HO2
21. H2 O2 + O −−→ HO2 + OH
Units: mol, cm,
2.30 × 1018
3.02 × 1012
4.79 × 1013
1.00 × 1013
7.08 × 1012
9.63 × 106
s, kJ, K.
-0.9
0.0
0.0
0.0
0.0
2.0
-7.1
5.8
33.3
15.0
6.0
2.0
Table A.1: The detailed H2 /O2 reaction mechanism from [54].
Appendices
A
Reaction mechanisms
The 21 reactions in the detailed hydrogen-oxygen mechanism are listed in table A.1 and the five of the
◦
reduced mechanism in A.2. The associated reaction rate is k = AT b e−E/R T .
B
B.1
Properties of S
Sparsity
As noted in § 3.2.3, S is often sparse. There is a way to determine which entries of S are identically zero using
the physical restrictions about how different species’ concentrations interact with each other. To determine
the sparsity in this way, each species X is characterized by a composite number ρX . First associate a prime
number pi with each atom type i = 1, . . . nα . Each species X is made up of a collection of atom types; let
ρX be the product of prime numbers corresponding to each type of atom making up species X. For example,
if elements H and O correspond to the prime numbers 2 and 3, then ρH = 2 and ρH2 O = 6. In effect, this
yields a prime number representation of each species where multiplicity is ignored.
42
Reaction
Hydrogen-oxygen chain
1. H + O2 −−→ OH + O
2. H2 + O −−→ OH + H
3. H2 + OH −−→ H2 O + H
A
b
E
3.52 × 1016
5.06 × 104
1.17 × 109
-0.7
2.7
1.3
71.4
26.3
15.2
Hydroperoxyl reactions
10. H + O2 + M −−→ HO2 + M
5.75 × 1019
12b. H2 + O2 −−→ HO2 + H
1.4 × 1014
Units: mol, cm, s, kJ, K.
-1.4
0.0
0.0
249.5
Table A.2: The reduced H2 /O2 reaction mechanism from [54].
Next, the columns of S correspond to chemical reactants and the rows to chemical products. The entry
sij controls how many atoms move from species j- a sort of reactant- to species i- a sort of product. The
operator can only move a positive amount of species Xj to Xi if the former contains all elements that comprise
the latter. If not, then the gcd(ρXi , ρXj ) < ρXi . But in this case there can be no flow of atoms from Xj to
Xi , and thus sij ≡ 0.
This technique can also be used to count the total number of entries that are identically zero in the
matrix. Call this total number Ω and, for i = 1, . . . , nS , let λi be the number of species Xj such that
gcd(ρXi , ρXj ) < ρXi . By the argument in the previous
paragraph, λi is the number of zeros in the ith column
PnS
of S. Then the number of zeros in S is Ω = i=1
λi because the sum is taken with respect to the different
species, and each of these correspond to a different column of S.
B.2
Non-positivity of eigenvalues
Enforcing the two constraints— (I) conservation of atoms and (II) non-negativity of concentrations yields
some interesting properties of the random matrix S. The non-positivity of eigenvalues is consistent with the
constraints: no species can grow arbitrarily large over time. The proof follows:
Theorem B.1. Let S be any random matrix such that ES = 0 and the off-diagonal elements of S be nonnegative. Then (a) the columns sum to zero, (b) the diagonal is negative, (c) the matrix is weakly diagonally
dominant, and (d) the eigenvalues are non-positive.
Proof. (a) Consider ES(·,j) = 0, where S(·,j) is the jth column of S. There are nα equations:
e1,1 s1,j + e1,2 s2,j + · · · + e1,nS snS ,j = 0
e2,1 s1,j + e2,2 s2,j + · · · + e2,nS snS ,j = 0
..
.
enα ,1 s1,j + enα ,2 s2,j + · · · + enα ,nP snS ,j = 0.
Now add the lines together:
XX
i
but
P
k ek,i
ek,i si,j = 0,
(143)
si,j = 0.
(144)
k
= 1 by definition. Thus,
X
i
43
(b) In equation (144), move the diagonal term to the RHS:
X
si,j = −sj,j .
(145)
i6=j
Since all off-diagonal terms are non-negative, it must be that the diagonal element is negative.
(c) The line above also shows weak diagonal dominance, since
X
si,j |sj,j | = i6=j
X
=
|si,j |,
(146)
(147)
i6=j
where the second equality holds because all off-diagonal elements are non-negative.
T
(d) Since
have the same eigenvalues, we will show that the claim is true for S T . Let B = S T
P S and S P
and Bi = j6=i |bi,j | = j6=i bi,j be the sum of off-diagonals in the ith row. Now let D(bi,i , Bi ) be the closed
disc centered at bi,i with radius Bi . Then the Gershgorin theorem states that every eigenvalue of B lies
within at least one of the discs [5]. In this case, we have bi,i = si,i and Bi = |si,i |, so every eigenvalue lies
within at least one disc D(si,i , |si,i |), where si,i ≤ 0.
References
[1] A
New
Templated
Implementation
https://github.com/libantioch/antioch.
of
Chemistry
Hydrodynamics
(Antioch).
[2] MIT Uncertainty Quantification Library (MUQ). https://bitbucket.org/mituq/muq.
[3] I. Babuska, F. Nobile, and R. Tempone. A systematic approach to model validation based on Bayesian
updates and prediction related rejection criteria. Computer Methods in Applied Mechanics and Engineering, 197:2517–2539, May 2008.
[4] M. J. Bayarri, J. O. Berger, R. Paulo, J. Sacks, J. A. Cafeo, J. Cavendish, C.-H. Lin, and J. Tu. A
framework for validation of computer models. Technometrics, 49(2):138–154, May 2007.
[5] H. E. Bell. Gershgorin’s theorem and the zeros of polynomials. American Mathematical Monthly,
72:292–295, 1965.
[6] M. L. Berliner. Hierarchical Bayesian time series models. In K. M. Hanson and R. N. Silver, editors,
Maximum Entropy and Bayesian Methods, volume 79 of Fundamental Theories of Physics, pages 15–22.
Springer Netherlands, 1996.
[7] C. E. Bonferroni. Il calcolo delle assicurazioni su gruppi di teste. In Studi in Onore del Professore
Salvatore Ortu Carboni, pages 13–60. Rome, 1935.
[8] G. E. P. Box and G. C. Tiao. Bayesian Inference in Statistical Analysis. Wiley-Interscience, 1 edition,
Apr. 1992.
[9] K. Braman, T. A. Oliver, and V. Raman. Bayesian analysis of syngas chemistry models. Combustion
Theory and Modelling, 17(5):858–887, 2013.
44
[10] M. P. Burke, M. Chaos, Y. Ju, F. L. Dryer, and S. J. Klippenstein. Comprehensive H2/O2 kinetic
model for high-pressure combustion. International Journal of Chemical Kinetics, 44(7):444–474, Dec.
2011.
[11] M. P. Burke, M. Chaos, Y. Ju, F. L. Dryer, and S. J. Klippenstein. Comprehensive H2/O2 kinetic
model for high-pressure combustion. International Journal of Chemical Kinetics, 44(7):444–474, July
2012.
[12] D. Calvetti and E. Somersalo. An Introduction to Bayesian Scientific Computing: Ten Lectures on
Subjective Computing, 2007.
[13] Center for Predictive Engineering and Computational Sciences. Quantification of Uncertainty for Estimation, Simulation, and Optimization (QUESO) User’s Manual, 2013.
[14] R. P. Christian. The Bayesian Choice. Springer, 2001.
[15] Conaire, H. J. Curran, J. M. Simmie, W. J. Pitz, and C. K. Westbrook. A comprehensive modeling
study of hydrogen oxidation. International Journal of Chemical Kinetics, 36(11):603–622, Nov. 2004.
[16] M. K. Cowles and B. P. Carlin. Markov chain Monte Carlo convergence diagnostics: A comparative
review. American Statistical Association, 91(434):883–904, June 1996.
[17] R. T. Cox. The Algebra of Probable Inference. Johns Hopkins University Press, 1961.
[18] M. H. DeGroot and M. J. Schervish. Probability and Statistics (4th Edition). Pearson, 4 edition, Jan.
2011.
[19] A. Edelman and N. R. Rao. Random matrix theory. Acta Numerica, 14(-1):233–297, May 2005.
[20] M. Feinberg. Lectures on Chemical Reaction Networks. 1979.
[21] A. Gelman. Exploratory Data Analysis for Complex Models. Journal of Computational and Graphical
Statistics, 13(4):755–779, Dec. 2004.
[22] A. Gelman, X.-l. Meng, and H. Stern. Posterior predictive assessment of model fitness via realized
discrepancies. In Statistica Sinica, pages 733–807, 1996.
[23] W. R. Gilks, S. Richardson, and D. Spiegelhalter. Markov Chain Monte Carlo in Practice. Chapman
and Hall/CRC, softcover reprint of the original 1st ed. 1996 edition, Dec. 1996.
[24] P. J. Green and A. Mira. Delayed rejection in reversible jump Metropolis-Hastings. Biometrika,
88(4):1035–1053, Dec. 2001.
[25] H. Haario, M. Laine, A. Mira, and E. Saksman. DRAM: Efficient adaptive MCMC. Statistics and
Computing, 16(4):339–354, Dec. 2006.
[26] H. Haario, E. Saksman, and J. Tamminen. An adaptive Metropolis algorithm. Bernoulli, 7(2):223–242,
2001.
[27] D. Higdon, J. Gattiker, B. Williams, and M. Rightley. Computer model calibration using highdimensional output. Journal of the American Statistical Association, 103(482):570–583, June 2008.
[28] D. Higdon, M. Kennedy, J. C. Cavendish, J. A. Cafeo, and R. D. Ryne. Combining field data and
computer simulations for calibration and prediction. SIAM Journal on Scientific Computing, 26(2):448–
466, Jan. 2004.
[29] E. T. Jaynes. Probability Theory: The Logic of Science. Cambridge University Press, 2003.
[30] J. Kaipio and E. Somersalo. Statistical and Computational Inverse Problems. Springer, 2005.
45
[31] M. C. Kennedy and A. O’Hagan. Bayesian calibration of computer models. Journal of the Royal
Statistical Society: Series B (Statistical Methodology), 63(3):425–464, Jan. 2001.
[32] A. A. Konnov. Remaining uncertainties in the kinetic mechanism of hydrogen combustion. Combustion
and Flame, 152(4):507–528, Mar. 2008.
[33] D. Lunn, A. Thomas, N. Best, and D. Spiegelhalter. WinBUGS - A Bayesian modelling framework:
Concepts, structure, and extensibility. 10(4):325–337, 2000.
[34] B. J. McBride, S. Gordon, and M. A. Reno. NASA Technical Memorandum 4513: Coefficients for
Calculating Thermodynamic and Transport Properties of Individual Species. Technical report, National
Aeronautics and Space Administration, 1993.
[35] M. L. Mehta. Random Matrices, Volume 142, Third Edition (Pure and Applied Mathematics). Academic
Press, 3 edition, Nov. 2004.
[36] K. Miki, S. H. Cheung, E. E. Prudencio, and P. L. Varghese. Bayesian uncertainty quantification of
recent shock tube determinations of the rate coefficient of reaction H + O2 OH + O. International
Journal of Chemical Kinetics, 44(9):586–597, July 2012.
[37] R. G. Miller. Simultaneous Statistical Inference (Springer Series in Statistics). Springer, 2nd edition,
Mar. 1981.
[38] A. Mira. On Metropolis-Hastings algorithms with delayed rejection. Metron, 59:3–4, 2001.
[39] R. D. Moser and T. A. Oliver. Validation of physical models in the precense of uncertainty. In
R. Ghanem, D. Higdon, and H. Owhadi, editors, Handbook of Uncertainty Quantification. Springer,
2016.
[40] M. A. Mueller, T. J. Kim, R. A. Yetter, and F. L. Dryer. Flow reactor studies and kinetic modeling of
the H2/O2 reaction. Int. J. Chem. Kinet., 31(2):113–125, Jan. 1999.
[41] T. A. Oliver, G. Terejanu, C. S. Simmons, and R. D. Moser. Validating predictions of unobserved
quantities. Computational Methods in Applied Mechanics and Engineering, 283:1310–1335, 2015.
[42] T. A. Oliver, G. Terejanu, C. S. Simmons, and R. D. Moser. Validating predictions of unobserved
quantities. Elsevier, submitted.
[43] E. Prudencio and S. H. Cheung. Parallel adaptive multilevel sampling algorithms for the Bayesian
analysis of mathematical models. International Journal for Uncertainty Quantification, 2(3):215–237,
2012.
[44] D. B. Rubin. More powerful randomization-based p-values in double-blind trials with non-compliance.
Statist. Med., 17(3):371–385, Feb. 1998.
[45] D. S. Sivia and J. Skilling. Data Analysis: A Bayesian Tutorial. Oxford University Press, 2006.
[46] G. P. Smith, D. M. Golden, M. Frenklach, N. W. Moriarty, B. Eiteneer, M. Goldenberg, C. T. Bowman,
R. K. Hanson, S. Song, W. C. Gardiner, V. V. Lissianski, and Q. Zhiwei. GRI-Mech. http://www.me.
berkeley.edu/gri_mech/. Accessed: 2015-01-15.
[47] C. Soize. Random matrix theory for modeling uncertainties in computational mechanics. Computer
Methods in Applied Mechanics and Engineering, 194(12-16):1333–1366, Apr. 2005.
[48] C. Soize. Generalized probabilistic approach of uncertainties in computational dynamics using random
matrices and polynomial chaos decompositions. Int. J. Numer. Meth. Engng., 81(8):939–970, Feb. 2010.
[49] J. I. Steinfeld, J. S. Francisco, and W. L. Hase. Chemical Kinetics and Dynamics. Prentice Hall, 1998.
46
[50] L. P. Swiler. Gaussian processes in response surface modeling. Society of Experimental Mechanics
Meeting, Jan. 2006.
[51] P. R. Westmoreland. Reduced kinetic mechanisms for applications in combustion systems. Edited by
N. Peters and B. Rogg, Spring-Verlag, New York, Lecture Notes in Physics, Monograph 15, 1993, 360
pp. AIChE J., 40(11):1926–1927, Nov. 1994.
[52] C. K. Wikle, R. F. Milliff, D. Nychka, and L. M. Berliner. Spatiotemporal hierarchical Bayesian modeling: Tropical ocean surface winds. Journal of the American Statistical Association, 96(454):382–397,
June 2001.
[53] F. A. Williams. Combustion Theory: Second Edition (Combustion Science and Engineering). Westview
Press, second edition edition, Mar. 1994.
[54] F. A. Williams. Detailed and reduced chemistry for hydrogen autoignition. Journal of Loss Prevention
in the Process Industries, 21(2):131–135, Mar. 2008.
47