Computational Aspects of Probability in
Non-Linear Mechanics
Hermann G. Matthies
Institute of Scientific Computing
Technische Universität Braunschweig
Brunswick, Germany
url: http://www.wire.tu-bs.de
Abstract Some ways on how probabilistic aspects may enter non-linear mechanics, and how this may be treated numerically are described. Deterministic systems with
stochastic excitation are well known, and here the emphasis is on uncertain systems,
where the randomness is assumed spatial. Traditional computational approaches usually use some form of perturbation or Monte Carlo simulation. This is contrasted here
with more recent methods based on stochastic Galerkin approximations.
Keywords: spatially stochastic systems, stochastic elliptic partial differential equations, stochastic Galerkin methods, Karhunen-Loève expansion, Wiener’s polynomial chaos, white noise analysis,
sparse Smolyak quadrature, Monte Carlo methods, stochastic finite elements
1
Introduction
Probabilistic or stochastic mechanics—e.g. [65]—deals with mechanical systems, which are
either subject to random external influences—a random or uncertain environment, or are
themselves uncertain, or both, cf. e.g. the reports [58, 79, 80, 88, 39]. Stochastic mechanics as
considered here is somewhat different from the mechanics of many particles, as investigated
in statistical physics. Although the mathematical methods may be similar, the goals are often
different. Also not considered will be probabilistic methods to solve deterministic problems
in mechanics, an area often just labelled as “Monte Carlo methods”, e.g. [77]. That term will
be used here as well, although in a more general meaning.
One may further make a distinction as to whether the mechanical system is modelled as
finite-dimensional—i.e. its time-evolution may be described by an ordinary differential equation (ODE) in time—or infinite-dimensional, e.g. a partial differential equation (PDE) with
spatial differential operators is used as a mathematical model. In the second case the numerical treatment involves also a discretisation of the spatial characteristics of the mechanical
system. Often the specific procedure is not important—but in case something specific has to
be taken, it is assumed here that the mechanical system has been discretised by finite elements, e.g. [98]. This then leads to an area often referred to as “stochastic finite elements”
(SFEM), e.g. [47, 58, 88], or “probabilistic/random finite elements”, e.g. [51]. This is the area
which will be dealt with in more detail in the following, and the reports [58, 79, 80, 88, 39]
contain more comprehensive surveys.
As the field is very large, this lecture tries to cover mainly new developments not yet
contained in text books, and necessarily reflects to some extent the preferences and work of
2
Hermann G. Matthies
the author. On the other hand, as the probabilistic ideas are sometimes not that well known
by workers in mechanics, it has to give a brief recapitulation.
2
Stochastics Models and Techniques
It will be assumed that the reader has some elementary knowledge of probability and stochastic processes, e.g. as contained in [72, 19, 28, 50, 52], cf. also [54, 63]. The text book [72]
also has an introductory description of in particular linear systems with (stationary) stochastic
input, an area quite well known and hence only addressed on the side here.
The main objective of the probabilistic aspects considered here—where they are not only
a computational device, e.g. [77], but part of the model, e.g. [54, 63]—is to make the uncertainty present in the model mathematically tractable and quantifiable [64, 21]. The first step
herein is to represent the uncertainty involved by means of probabilistic methods. This is not
the only possibility, fuzzy sets or convex models, cf. e.g. [55, 20], are other ones. This lecture
will adhere to the probabilistic point of view.
The description of uncertainty—of which there are different kinds, the most important
distinction being between aleatoric uncertainty, which is regarded as inherent in the phenomenon, and epistemic uncertainty, which results from our incomplete knowledge—is usually accomplished by representing certain quantities as random variables, stochastic processes, or random fields. Elementary probability and random variables assumed to be known,
in the following a short recap on the aspects of stochastic processes and random fields in
mechanics which are important for the computational approaches will be given.
2.1
Stochastic Processes and Random Fields
In what follows, it will be useful to consider a set Ω of random elementary events, together
with a class of subsets A of Ω—technically a σ-algebra e.g. [50, 52, 19, 28]—to which a real
number in the interval [0, 1] may be assigned, the probability of occurrence—mathematically
a measure P . A V-valued random variable r is then a function relating to each ω ∈ Ω an
element r(ω) ∈ V. Mostly V is a vector space, usually the real numbers R—some technicalities have to be added, such that r is measurable. Viewing random variables and other
random elements as functions on Ω will help to make a connection later to other well-known
approximation processes.
A random variable r—for the sake of simplicity assume that V = R—is considered completely specified by its distribution function [50, 52, 72, 19, 28]
Z
∀r ∈ R : Fr (r) = Pr{r(ω) ≤ r} :=
dP (ω).
(1)
{r(ω)≤r}
The simplest characterisation of a random variable involves its mean, or average, or expected value E (r), and the splitting into the mean r̄ and the fluctuating part r̃,
r(ω) = r̄ + r̃,
with E (r̃) = 0,
Z
r(ω) dP (ω),
r̄ := E (r) :=
(2)
(3)
Ω
as well as the covariance
Cr := E (r̃ ⊗ r̃) .
(4)
3
Computational Aspects of Probability
If the cross-covariance Cr1,2 = E (r̃1 ⊗ r̃2 ) of two random variables r1 and r2 vanishes, they
are called uncorrelated.
Consider a time interval T = [0, T ]—often the interval (−∞, +∞) is considered, a
stochastic process then is a prescription of a random variable st at each instance t ∈ T,
or alternatively the assignment of a function sω (t) on T—this is called a realisation—from
some class F(T) to each ω ∈ Ω, or a function of two variables s(t, ω) with values in V—all
these are equivalent modulo some technical details, cf. [50, 52, 72, 39].
A stochastic process—assume again that V = R—is considered completely specified
(modulo some technicalities) by its finite dimensional (fi-di) distributions [50, 52, 72, 19, 28],
i.e. for each n ∈ N and
∀t1 , . . . , tn ∈ T, ∀s1 , . . . , sn ∈ R :
Fs (t1 , . . . , tn ; s1 , . . . , sn ) = Pr{s(t1 ) ≤ s1 , . . . , s(tn ) ≤ sn }
(5)
is given. Often this much information is not available, and one may only know the socalled second order information—mean and covariance, and the marginal resp. pointwise
distribution—this is Eq. (5) for n = 1.
Similarly to a random variable, a stochastic process may be conveniently split into its
mean s̄(t)—now a function of t—and the fluctuating part s̃(t, ω), which has zero mean. The
covariance may now be considered at different times to give the covariance function
Cs (t1 , t2 ) := E (s̃t1 ⊗ s̃t2 ) .
(6)
In case the random properties of s(t, ω) are independent of t, the process is called strictly
stationary or strict sense stationary (SSS) [72]; and in case of only the mean s̄ being independent of t, i.e. constant, and the covariance then a function of the time lag, i.e. Cs (t1 , t2 ) =
cs (t1 − t2 ), the process is called weakly stationary or weak sense stationary (WSS). In this
case the process is often characterised by its spectrum Ss (ν), the Fourier transform of the
time-lag covariance function
Z T
k
Ss (νk ) =
cs (t) exp(−i2πνk t) dt, νk = ; k ∈ Z,
(7)
T
0
with obvious modifications if the time interval T is infinite [9]. In this case the process may
be synthesised from its spectrum, e.g. [83]:
s(t, ω) = s̄(t) +
∞
X
p
ςk (ω) Ss (νk ) exp(i2πνk t),
(8)
k=−∞
where the ςk (ω) are uncorrelated random variables of unit variance and vanishing mean [52],
the series converging in L2 (T × Ω).
The description of random fields [1, 91, 92, 11, 26, 83], where we assign a random variable r(x, ω) to each point x ∈ R in some region R in space, is in many ways very similar
to the case of stochastic processes. The only difference is that the “index set” has changed
from the interval T to the region in space R. The decomposition into mean and fluctuating
part with zero mean of Eq. (2) is as before, and so is the definition of the covariance function
Eq. (6). What was called stationary before is now called homogeneous. A realisation—the
spatial field r(·, ω) for one specific ω—of a positive non-Gaussian random field κ(x, ω) on an
L-shaped domain R ⊂ R2 is shown in Fig. 1.
4
Hermann G. Matthies
Figure 1: Realisation of κ.
In case the region R is a product of intervals, a multi-dimensional spectrum may be computed, but most space-regions of interest are not as simple. This necessitates the generalisation
in the next section 2.2 of the Fourier expansion in Eq. (7), which will then be valid also for
non-stationary or non-homogeneous stochastic processes or random fields.
2.2
Karhunen-Loève Expansion
What is desired is a decomposition similar to the spectral synthesis Eq. (8) into products of deterministic functions on the region—in Eq. (8) the exponentials—and functions only dependent on ω, i.e. simple random variables. This is furnished by the Karhunen-Loève expansion
[52], also known as proper orthogonal decomposition, and may also be seen as a singular
value decomposition [30] of the random field. Given the covariance function Cr (x1 , x2 )—
defined analogous to Eq. (6)—one considers the Fredholm eigenproblem [14, 76]:
Z
Cr (x1 , x2 )φk (x2 ) dx2 = %2k φk (x1 ), x1 ∈ R.
(9)
R
This equation can usually not be solved analytically, but standard numerical methods [2] (e.g.
based on the FEM) may be used to discretise and resolve this Fredholm integral equation.
Examples of two eigenfunctions for the field of Fig. 1 are shown in Fig. 2.
As the covariance function is—through its definition Eq. (6) —symmetric and positive
semi-definite, Eq. (9) has positive decreasingly ordered eigenvalues {%2k }k=1,∞ with only ac-
5
Computational Aspects of Probability
Eigenfunction No. 1
Eigenfunction No. 15
KL−mode 15
KL−mode 1
1
1
0.8
0.5
0
0.6
−0.5
0.4
−1
0.2
−1.5
0
1
−2
1
−1
0.5
−1
0.5
−0.5
0
−0.5
0
0
−0.5
0
−0.5
0.5
−1
0.5
−1
1
1
Figure 2: KL-Eigenfunctions Nos. 1 and 15.
cumulation point zero, and a complete and orthonormal—in L2 (R)—set of eigenfunctions
{φk }. This yields the spectral decomposition
X
Cr (x1 , x2 ) =
%2k φk (x1 ) ⊗ φk (x2 ).
(10)
k
This may be compared with an expression associated with Eq. (7)
X
X
Cs (t1 , t2 ) = cs (t1 − t2 ) =
Ss (νk )ei2πνk (t1 −t2 ) =
Ss (νk )ei2πνk t1 e−i2πνk t2
k
k
which is the complete analogue of Eq. (10) when we observe that the two exponentials—
which are conjugate complex of each other, something not needed in Eq. (10) as the eigenfunctions there are real—may be separated into sines and cosines, and that the spectral values
Ss (νk ) are the eigenvalues of the—with a covariance function only dependent on the time
lag—convolution operator with kernel cs (t1 − t2 ) [14, 76, 86], the exponentials or the sines
and cosines being its eigenfunctions. We see this correspondence also in Fig. 2, where higher
order eigenfunctions display higher spatial “frequencies” or wave-numbers.
This gives us similarly to Eq. (8) a possibility to synthesise the random field through its
Karhunen-Loève expansion (KLE)—or proper orthogonal decomposition (POD), or singular
value decomposition (SVD). Two such truncated expansions for the field in Fig. 1 with the
eigenfunctions from Fig. 2 are shown in Fig. 3:
X
r(x, ω) = r̄(x) +
%k ρk (ω)φk (x),
(11)
k
where again ρk (ω) are uncorrelated random variables with unit variance and zero mean
E (ρk ρm ) = δkm . This is a reflection of the orthonormality of the eigenfunctions, uncorrelated is the stochastic pendant to orthogonal.
In case the stochastic process s(t, ω) or the random field r(x, ω) is Gaussian [50, 52, 72,
39], the random variables ςk (ω) in Eq. (8), and the random variables ρk (ω) in Eq. (11) are
also Gaussian—a linear combination of Gaussian variables is again Gaussian. As they are
uncorrelated, they are also independent [52], and the synthesis is particularly simple. NonGaussian fields may be synthesised as functions of Gaussian fields, e.g. [31, 25, 39, 46] and
6
Hermann G. Matthies
Realisation with 20 KL-terms
Realisation with 50 KL-terms
6
6
4
4
2
2
0
1
0
1
1
0.5
0
0.5
0
0
−0.5
1
0.5
0.5
0
−0.5
−0.5
−1 −1
−0.5
−1 −1
Figure 3: Realisations of κ with 20 and 50 KL-Terms.
the references therein, i.e.
r(x, ω) = Ψ(x, g(x, ω));
X
g(x, ω) =
γk θk (ω)ψk (x),
(12)
(13)
k
where Ψ is a pointwise—usually non-linear—transformation, and g(x, ω) is a Gaussian field
with uncorrelated and hence independent Gaussian variables θk (ω), the ψk being the KLEfunctions of g. Another possibility is explored in the next section 2.3, cf. [91, 83, 78].
2.3
Polynomial Chaos Expansion
As suggested by Wiener [93], any random variable r(ω)—subject to some restrictions of a
technical nature—may be represented as a series of polynomials in uncorrelated and independent Gaussian variables θ = (θ1 , . . . , θ , . . .) [50, 26, 31, 34, 56, 35, 58, 25, 59, 32, 94,
39, 62], the polynomial chaos expansion (PCE):
X
r(ω) =
r(α) Hα (θ(ω)).
(14)
α∈J
The word chaos was used here by N. Wiener [93], and has nothing to do with modern usage
of the word in mathematics, where it characterises the unpredictable behaviour of dynamical
systems. Here J := {α | α = (α1 , . . . , α , . . .), α ∈ N0 } is the set of infinite multi-indices,
i.e. sequences of non-negative integers, only finitely many of which are non-zero [34, 56, 59,
39, 62]. The r(α) are the generalised Fourier coefficients for the orthogonal basis of Hermite
polynomials (two are shown in Fig. 4):
Hα (θ(ω)) =
∞
Y
=1
hα (θ (ω)),
(15)
7
Computational Aspects of Probability
H(2,3)
H(3,3)
6
4
4
2
2
0
0
−2
−2
−4
−6
2
−4
2
1
0
−1
−2 −2
0
−1
1
2
1
0
−1
−2 −2
y
0
−1
1
2
y
Figure 4: Hermite Polynomials H(2,3) and H(3,3) .
where the h` (ξ) are the usual Hermite polynomials, and θ(ω) := (θ1 (ω), . . . , θ (ω), . . . ) is
an infinite sequence of normalised independent Gaussian random variables.
Orthogonality is
Q
reflected in hHα |Hβ i := E (Hα (θ)Hβ (θ)) = α! δα,β , where α! := ∞
α
!.
=1 
If the random variable r(ω) is in L2 (Ω), i.e. has finite variance, the series Eq. (14) converges in that norm, i.e. in variance. This expansion may now be used for the random variables
ςk (ω) in Eq. (8), and the random variables ρk (ω) in Eq. (11). This then gives a convenient
representation in independent identically distributed (iid) Gaussian variables of the stochastic
process resp. random field:
r(x, ω) = r̄(x) +
∞ X
X
(α)
%k rk Hα (θ(ω))φk (x).
(16)
k=1 α∈J
One should point out once more that the approach described here is one possibility to
discretise random fields in the context of the stochastic finite element method, see the reports
[58, 83, 79, 80, 88, 39] and the references therein for other possibilities.
3
Overview on Computational Approaches
As there is not enough time and space in this lecture to describe the multitude of computational approaches for stochastic problems in detail, a short overview on the main techniques will be given here. Subsequently, the lecture will concentrate on just one approach,
the stochastic Galerkin methods in section 4.
Stochastic systems can be interpreted mathematically in several ways. At the moment
we concentrate on randomness in space. If evolution with stochastic input varying in time
has to be considered, one may combine the techniques described here with the already well
established methods in that field [48, 22]; for theoretical results, e.g. see [7, 28, 38, 70]. There
is also plenty of text book literature for the methods working in the Fourier domain, e.g. see
[5, 9, 13, 22, 38, 50, 72]. One should also mention approaches to compute directly moments
8
Hermann G. Matthies
or other functionals of the solution in the case of evolutionary stochastic ordinary differential
equations [84].
More information, references, and reviews on stochastic finite elements can be found in
[58, 79, 88, 39]. A somewhat specialised field is the area of structural reliability, e.g. see
[3, 17, 33, 54, 63].
3.1
Monte Carlo Simulation and Other Direct Integration Methods
Both Monte Carlo Simulation [10, 71, 81] and its faster cousin Quasi Monte Carlo [10, 66]
are best viewed as integration techniques. When solving for the response of a stochastic
system, the ultimate goal is usually the computation of response statistics, i.e. functionals of
the solution, or more precisely expected values of some functions of the response, such as
Z
Ψu (x) = E (Ψ(x, ω, u(x, ω))) =
Ψ(x, ω, u(x, ω)) dP (ω).
(17)
Ω
Such an integral may numerically be approximated by a weighted sum of samples of the
integrand.
Monte Carlo (MC) methods can be used directly for this, but they require a high computational effort [10]. Variance reduction techniques are employed to lower this effort somewhat.
Quasi Monte Carlo (QMC) methods [10, 66] may reduce the computational effort considerably without requiring much regularity. But often we have high regularity in the stochastic
variables, and this is not exploited by QMC methods. Monte Carlo chooses the samples randomly according to the underlying probability measure, and the weights all equal, e.g. [10].
Quasi Monte Carlo, e.g. [10, 66] chooses deterministic low-discrepancy evaluation points
and still all weights equal. Common Quadrature rules choose special evaluation points and
corresponding weights, .e.g. [85, 23, 67, 68, 74, 75].
If the aim is to compute a functional of the solution such as in Eq. (17), one would try
to approximate the integral numerically. First one approximates the “infinite dimensional”
measure space Ω by a “finite dimensional” one, Θm :
Ψu (x) ≈
Z
X
wz Ψ(x, θ z (ω), s(x)u(θ z (ω))),
(18)
z=1
where s(x) are the spatial FEM-basis such that s(x)u(θ(ω)) is the approximate FE-solution,
θ z ∈ Θm are the evaluation points, and wz are the weights—in MC- and QMC methods the
weights are wz = 1/Z.
To compute this integral, proceed in the following way:
1. Select the points {θ z |z = 1, . . . , Z} ⊂ Θm according to the integration rule.
2. For each θ z —a realisation of the stochastic system—solve the deterministic problem
with that fixed realisation, yielding u(θ z ).
3. For each θz compute the integrand Ψu (x, θ z , s(x)u(θ z )). in Eq. (18).
4. Perform the summation in Eq. (18).
Computational Aspects of Probability
9
Note the potential for “embarrassingly easy” parallelisation in steps 2 and 3.
The Quasi Monte Carlo method is only slightly different from this point of view, as all
the difference is that the points {θ z } are selected from a “low discrepancy sequence”, see
[66, 10].
Monte Carlo is very robust, and almost anything can be tackled by it. But on the other
hand it is very slow, and this is the main disadvantage. Its other main advantage is that it is
not affected by the dimension of the integration domain as most other methods are. Sparse
(Smolyak) quadrature methods are an efficient alternative. These have initially been described
in [85], and have found increasing attention in recent years, e.g. [67, 75, 23].
As integrals over the probability space are also part of most other approaches, these integration methods are usually found inside many other techniques.
If the purpose of the computation of the expectation is to calculate the probability of rare
events—like in the evaluation of reliability—there are special methods like FORM or SORM
specifically tailored for this task, e.g. [3, 17, 33, 54, 63].
3.2
Perturbation Techniques
Alternatives to Monte Carlo (e.g. [71, 18]) methods, which compute the first moments of
the solution have been developed in the field of stochastic mechanics—cf. [51], for example
perturbation methods, e.g. [47], or methods based on Neumann-series, e.g. [26, 4].
The perturbation approach—e.g. see [47, 58]– does not really start from a probabilistic
consideration, but from expecting the stochastic response to be a small variation about the
mean. Assuming the stochastic parameters to vary a little bit around their mean, a formal
perturbation expansion—usually only up to first or second order—is performed, which gives
the variation of the response about the mean. By taking expected values one gets approximate
expressions for the most basic response quantities, the mean and the covariance.
As may be expected, the perturbation approach usually only works well when the stochastic variations are not too large. It becomes important in another guise though, namely in its
use as an error estimator. Here it is then known as the adjoint or dual method. It may also be
used in a stochastic context, see [43] and the references therein, also to the usual use as error
estimator.
3.3
Response Surfaces
Another possibility [49] in stochastic computations is to evaluate the desired response in
some—maybe well chosen—points of the stochastic parameter space, and then to fit these
response points with some analytic expression—often a low order polynomial. This then is
the response surface, which may be taken as a substitute of the true response. All subsequent
computations can then be done with this response surface, e.g. [3, 33, 54, 63] . Current fitting techniques usually only work well for low order and not too many dimensions of the
stochastic parameter space.
3.4
Galerkin Methods
Following [26], stochastic Galerkin methods have been applied to various linear problems,
e.g. [24, 25, 73, 61, 40, 37, 94, 95]. Recently, non-linear problems with stochastic loads have
10
Hermann G. Matthies
been tackled, e.g. [95], and some first results of both a theoretical and numerical nature for
non-linear stochastic operators are in [41, 44].
These Galerkin methods allow us to have an explicit functional relationship between
the independent random variables and the solution—in contrast with usual Monte Carlo
approaches—so that subsequent evaluations of functionals—statistics like the mean, covariance, or probabilities of exceedance—are very cheap. This may be seen as a way to systematically calculate more and more accurate “response surfaces” [49]. They are described in more
detail in the next section 4.
4
Stochastic Galerkin Methods
It is quite useful to realise that quite similar problems may also arise in other areas of science
and engineering, as this gives a chance for cross-fertilisation and subsequent faster development. The connection to mathematical methods of statistical physics was already mentioned,
other important areas are electrical engineering, signal processing, filtering, and control.
As in many other fields, it is good to have some way of classifying the computational task
at hand according to the level of difficulty etc. A simple classification will be attempted here.
Consider a stochastic process or random field f as input to a system given by B, where the
output or response u has to satisfy an equation:
B(u) = f.
(19)
The first distinction is whether the system B is deterministic, i.e. known with certainty, or
whether it is also uncertain and hence stochastic. Deterministic systems with stochastic or
random inputs have been considered extensively in many fields of science and engineering,
e.g. [72, 9, 13, 28, 32, 38, 47, 48, 50, 62, 70, 79, 84], where this case—particularly in the
literature on stochastic differential equations—is sometimes referred to as additive, or external noise [53]. In this case the response u may be stochastic just due to the excitation f being
so. More challenging are systems with multiplicative, or internal resp. parametric noise or
uncertainty. This may either come about through a direct interaction between the state of the
system and the noise, or the system may just be uncertain. In this case the response u may be
stochastic just because the system B is so, even with deterministic input f.
The second distinction is whether the system is memory-less in a general sense (e.g. [72]),
i.e. it feels nothing of the temporal or spatial characteristics of the stochastic process or random field. These systems are given by simple transformations such as Eq. (12). Examples
from electrical engineering are [72] the square-law detector, the rectifier, or the hard-limiter.
Such transformations are sometimes used artificially in order to realise non-Gaussian fields
and processes [97, 46].
For the sake of simplicity then, this lecture will focus on stationary systems, these being usually described by elliptic linear or non-linear partial differential equations. Here we
have stochastic coefficients, and the equations are labelled as stochastic partial differential
equations (SPDEs).
One may start with the simplest case, a linear partial differential equation where just
the right-hand-side (RHS) or source term is a random field, something which may be called
additive noise. As the best known approach the Monte Carlo method and some variants are
reviewed first, so that the Galerkin method may be put into perspective. The next step in
difficulty is still the linear partial differential equation, but with a random field in the operator.
11
Computational Aspects of Probability
This may be termed as multiplicative noise. It is formally linear, but in the stochastic response
it is non-linear. With this preparation, stochastic Galerkin methods for general non-linear
elliptic boundary value problems may be described.
To fix ideas, assume that the following equation has to be solved:
−∇T (κ(x, ω)∇u(x, ω)) = f (x, ω) for x ∈ R,
u(x, ω) = 0 for x ∈ ∂R
(20)
for (almost) all ω ∈ Ω. It can be thought of as a model for a membrane and its vertical
displacement u(x, ω) under tension κ(x, ω) and load f (x, ω), e.g. on a drum. Or it may be
thought of as a model of a diffusive process, such as heat flow, where then u(x, ω) is the
temperature, κ(x, ω) the heat conductivity, and f (x, ω) heat sources. Another interpretation
in this direction is the flow of groundwater in the subsurface, where u(x, ω) is the hydraulic
head, κ(x, ω) the flow conductivity, and f (x, ω) are sources or sinks. As written in Eq. (20),
the problem is linear, but later the coefficient κ(x, ω) will be assumed to depend on the solution u(x, ω), making the problem non-linear. In any case it is required that
0 < κ− < κ(x, ω) < κ+ < ∞,
(21)
as otherwise the problem is not well-posed [62].
The so-called strong form in Eq. (20) is not a good starting point for the Galerkin approach
[26], and as in the purely deterministic problem [98, 87, 12, 86] a variational formulation is
needed [62]: Look for the solution u in a space V ⊗ S of functions which have a spatial
dependence (the variables from V), and a stochastic dependence (the variables from S), such
that for all w ∈ V ⊗ S:
Z Z
b(u, w) :=
(∇w(x, ω))T κ(x)(∇u(x, ω)) dx dP (ω) = hhB(u), wii
Ω R
Z Z
f (x, ω)w(x, ω) dx dP (ω). (22)
hhf, wii :=
Ω
R
The form b(u, w) defines the operator B, and is always linear in w, and for a linear problem
it is also linear in u. In the interpretation as a membrane, Eq. (22) is an averaged statement
of virtual work. Taking the decomposition Eq. (2) u(x, ω) = ū(x) + ũ(x, ω), assuming that
in the simplest case the form b(u, w) is not only linear in u but also κ is not stochastic, one
sees that Eq. (22) in this simple case decomposes into two equations:
∀v ∈ V :
∀w ∈ V ⊗ S :
b(ū, v) = hhf¯, vii,
b(ũ, w) = hhf˜, wii.
(23)
(24)
This is the case to be treated in the next section 4.1, and one sees that Eq. (23) is a purely
standard deterministic problem [15, 82, 62].
4.1
Linear Systems with Stochastic Right Hand Side—Additive Noise
This kind of situation is the simplest of the stochastic problems, and it allows us to introduce
our approach at an easy pace. It also is most similar to what is common knowledge about
linear time-invariant dynamical systems with random (time varying) inputs. There we can
12
Hermann G. Matthies
use the Fourier transform, and then it is essentially the transfer function of the system—or
rather the square of the transfer function—which maps the spectrum of the input process
onto the spectrum of the output. A similar situation will be observed here, but the Fourier
transform has to be generalised to the Karhunen-Loève expansion.
D ISCRETISATION AND N UMERICAL A PPROXIMATION : Assume that the spatial part
of the SPDE has been approximated by a Galerkin method—here the finite element method
(FEM). In some sense an arbitrary spatial discretisation could be used, but as we deal with
Galerkin methods in the stochastic domain, assuming this also in the spatial domain gives
a certain measure of unity to the presentation. What is used in the discretisation amounts to
the finite element method in space—it does not matter which variant—and a spectral or pure
p-method in the stochastic dimension [26].
S PATIAL D ISCRETISATION : Performing a Galerkin approximation in the spatial part
amounts to taking only a finite-dimensional subspace VN ⊂ V. Let {s1 (x), . . . , sN (x)} be a
basis in VN , we then approximate the solution by [12, 87, 98]
u(x, ω) =
N
X
sk (x)uk (ω) = s(x)u(ω),
(25)
k=1
where the {uk (ω)} now are random variables in S, and for conciseness of notation we have
set s(x) = [s1 (x), . . . , sN (x)] and u(ω) = [u1 (ω), . . . , uN (ω)]T .
Inserting this ansatz into Eq. (22), and applying the spatial Galerkin conditions, we then
require that for all ϕ ∈ S :
Z
Z
Z
ϕ(ω)f̃ (ω) dP (ω),
(26)
ϕ(ω)ũ(ω) dP (ω) =
ϕ(ω)K ũ(ω) dP (ω) = K
Ω
Ω
Ω
R
where K = (Kı ) = R (∇sı (x))T κ(x)(∇s (x)) dx is the usual deterministic stiffness
R
matrix, and the RHS is f̃ (ω) = [f˜1 (ω), . . . , f˜N (ω)]T with f˜ (ω) = R s (x)f˜(x, ω) dx. The
variational Eq. (26) will be written as K ũ(ω) = f̃ (ω), understood in a weak sense. This
involves the variable ω ∈ Ω, and is still computationally intractable, as in general we need
infinitely many coordinates to parametrise Ω.
S TOCHASTIC D ISCRETISATION : In section 2.3 we used the PCE to represent a stochastic
process. Here we want to extend the Galerkin idea, and for that reason we expand the random
variables ũ(ω)T = [ũ1 (ω), . . . , ũN (ω)] in a PC-series:
X (α)
∀k : ũk (ω) =
uk Hα (ω) = uk H(ω),
(27)
α
where H(ω)T = [. . . , Hα (ω), . . .], and uk = [. . . , uαk , . . .].
For the purpose of actual computation, truncate the PCE Eq. (27)) after finitely many
terms α ∈ JM,p , thus introducing a finite dimensional approximation span{Hα |α ∈ JM,p } ⊂
S. The set JM,p ⊂ J is here defined for M, p ∈ N as (see also [62]):
JM,p = {α ∈ J |∀ > M : α = 0, |α| ≤ p},
where
|α| :=
∞
X
α .
(28)
=1
S
Just as we require
S for N VN to be dense in V —see [87, 12] —here we rely on the fact
that the closure of M,p span{Hα |α ∈ JM,p } is all of S, see [62].
13
Computational Aspects of Probability
C OMPUTATIONAL P ROCEDURES : Even after the discretisation has been spelled out in
the preceeding section, there still remain many choices on how to actually organise the computation. Some of them may be seen as discretisations in their own right, or when applied
to an already finite dimensional problem they could be more appropriately labeled as methods to reduce the dimensionality of the system. As usually stochastic problems are addressed
with the Monte Carlo method, we give here a quick summary, and we use some of these ideas
later in a different setting. Subsequently, we show different ways of making the approximation of the stochastic part; these are the pure polynomial chaos expansion (PCE), and the
Karhunen-Loève expansion (KLE) of the RHS.
Monte Carlo Methods: Let us touch shortly on the Monte Carlo (MC) method, as it would
be often the natural first choice for anyone faced with solving a stochastic PDE.
In the execution of the algorithm in section 3.1, the problem is that we do not know u(ω),
as this is the still unknown solution. But Eq. (18) does not really require that one knows u(ω)
for all ω, i.e. to have a functional expression for it. All one has to know is the solution at
the particular evaluation points θ z , which means the solution for particular realisations of the
random field, in this case f (x, θ z ).
Note that this form of computation is completely independent of what kind of stochastic
problem we have, be it linear or non-linear, or with stochastic RHS or stochastic operator.
This is on one hand the strength of this approach—its general applicability, but on the other
hand it does not take into account any special properties of the problem.
Using the Polynomial Chaos Expansion: A PCE similar to Eq. (14) allows us to write
Eq. (26) with an expansion of both ũ and f̃ (see [26, 62]):
X
X
K ũ(ω) =
Ku(α) Hα (ω) = f̃ (ω) =
f (α) Hα (ω).
(29)
α∈J
α∈J
As
is unique, we have that the coefficients u(α) in the PCE of ũ(ω) =
P the representation
(α)
Hα (ω) satisfy for all α ∈ J the uncoupled system of equations Ku(α) = f (α) .
α∈J u
This means that the coefficients in the PCE may be computed one-by-one and independently
of each other—and hence in parallel with no interaction. A similar situation is familiar from
linear time-invariant systems under stochastic loading and computation in the frequency domain: Each frequency may be solved for independently. Here this is due to the orthogonality
of the polynomial chaos basis. As the set J is infinite, in an actual computation we take a
finite subset such as JM,p ⊂ J defined in relation Eq. (28), and compute only those u(α) with
α ∈ JM,p . Once we have the u(α) , we may compute approximately any statistic Eq.(17) of u,
and especially – ū being already approximated by ū —the covariance
Cu (x, y) ≈ s(x)E (ũ(ω) ⊗ ũ(ω)) sT (y) = s(x)C u sT (y),
P
where C u := E (ũ(ω) ⊗ ũ(ω)) = α∈JM,p u(α) ⊗ u(α) is the discrete covariance matrix of
ũ(ω).
Remark 1. We could have used the PCE already in the continuous case
X
X
u(x, ω) =
u(α) (x)Hα (ω),
f (x, ω) =
f (α) (x)Hα (ω);
α
α
—where the coefficients are given by f (α) (x) := E (f (x, ·)Hα (·)) —and we would have seen
that each u(α) (x) satisfies the same PDE Eq. (23), but with different (deterministic) RHS
14
Hermann G. Matthies
f (α) (x). The f (α) (x) may be smoother than f (x, ω) and hence a coarser spatial discretisation
could suffice, or the discretisation could be different for each α. For the sake of simplicity we
shall not explore this possibility further and assume that the spatial discretisation is always
the same.
Certainly the f (α) (x) or f (α) have to be given or computed somehow, and a potential
problem is that one may not have an explicit expression for f (x, ω) with which to compute
it. Also, in the discrete case, there can be only N linearly independent f (α) — N = dim VN
is the number of spatial basis functions—so one ought to do better.
Using the Karhunen-Loève Expansion: The Karhunen-Loève expansion (KLE) opens
up another possibility. In the spatially discretised setting the eigenvalue problem Eq. (9) and
spectral decomposition Eq. (10) looks like
C f f ` = λ` f ` ,
C f = Φf Λf ΦTf ,
(30)
with Φf = [. . . , f ` , . . .], and Λf = diag(λ` ). Writing the already spatially discretised KLE
P
of the RHS f̃ (ω) = ` ϕ` (ω)f ` = Φf ϕ(ω) with ϕ(ω) = [. . . , ϕ` (ω), . . .]T , we insert this
into Eq. (26), and obtain
X
ϕ` (ω)K −1 f ` .
(31)
ũ(ω) = K −1 f̃ (ω) = K −1 Φf ϕ(ω) =
`
This implies that each u(α) in Eq.(29) may be computed as a linear combination of the vectors
û` , satisfying K û` = f ` , which may be computed independently from each other:
X (α)
ϕ` û` ,
u(α) = E Hα (·)K −1 Φf ϕ(ω) = K −1 Φf ϕ(α) =
`
(α)
(α)
where ϕ(α) = [. . . , ϕ` , . . .]T and ϕ`
= E (Hα (·)ϕ` (·)).
Remark 2. A remark similar to Remark 1 applies here as well: The continuous analogues
of f ` , the KL-eigenfunctions of Cf (x, y), are often much smoother than f itself; hence the
spatial discretisation may be reduced, or even better adapted individually for each f ` . Again,
for the sake of simplicity of exposition, we assume that the spatial discretisation is always the
same.
Looking at Eq. (16), it is obvious that for each KL-term there will be at least one PCterm, so there are less solves in Eq. (31) for the KLE than for the pure PCE. If one uses all
N eigenvectors, there is no difference to the solution in the previous subsection, other than
that the computations are arranged differently. But the idea is to have many fewer vectors.
A problem is that the choice based on the spectrum of Cf (x, y) alone does not tell us how
important they are in the solution u(ω), as we have not taken account of the transfer properties
of the system. Best would be of course to use the KLE of the solution u(ω). An approach for
this is shown in [62].
4.2
Linear Systems with Stochastic Operator—Multiplicative Noise
Going up one level higher in the difficulty, consider a linear partial differential equation
Eq. (22) with a stochastic coefficient, satisfying the relation in Eq. (21). A standard spatial
discretisation like in Eq. (26) leads to a problem of the form
K(ω)u(ω) = f (ω),
(32)
15
Computational Aspects of Probability
where both the matrix K(ω) and the RHS f (ω) are random variables, and hence so is u(ω).
The Eq. (32) may be termed as one with both additive and multiplicative noise. Again expand
the solution u(ω) as a linear combination of Hermite polynomials in ω
X
u(ω) =
u(α) Hα (ω),
(33)
α
and the task is to compute the u(α) .
Often it is justified to assume that the random variables K(ω) and f (ω) are independent,
hence Ω = Ω1 × Ω2 , and K(ω) is a random variable only on Ω1 , whereas f (ω) is a random
variable only on Ω2 , so that K(ω1 )u(ω1 , ω2 ) = f (ω2 ). The solution in that case may in a
more refined way be written as
XX
u(ω1 , ω2 ) =
u(α1 ,α2 ) Hα1 (ω1 )Hα2 (ω2 ).
(34)
α1
α2
P
(α1 ,α2 )
Introducing the u(α1 ,·) (ω2 ) =
Hα2 (ω2 ), one may see [62] that each satisfies
α2 u
an equation of the type where only the RHS is stochastic, a problem just considered in the
previous section 4.1 on additive noise or stochastic
RHS. This we now know how to handle,
P
and hence concentrate on u(·,α2 ) (ω1 ) = α1 u(α1 ,α2 ) Hα1 (ω1 ) = K(ω1 )−1 f (α2 ) , which for
each α2 is a system with purely multiplicative noise [62], i.e. where the RHS is deterministic;
and on this we focus now: K(ω)u(ω) = f .
C OMPUTATIONAL P ROCEDURES : Similarly as in section 4.1, different computational
approaches are explored. As now also the operator is approximated, not just by projecting it
onto a finite dimensional subspace, but by using the KL- and PC-expansions on the coefficient
functions, one has to be concerned with stability of the numerical approximation; and with
the Monte Carlo method, some care has already to be exercised in order to preserve stability
of the numerical approximation process.
Monte Carlo Methods—Continued: In principle one could proceed like before in section 4.1. Generate samples θ z , solve the equations etc., as described before. One aspect deserves further consideration though: The realisations of the conductivity all have to satisfy
the relation Eq. (21). Generating the realisations through either the pure PCE Eq. (14), or
through the combined KL- and PC-expansion Eq. (16), to be numerically feasible it has to be
truncated to a finite number of terms. But as both expansions only converge in L2 (Ω), there is
no guarantee that for a fixed θ z the relation Eq. (21) can be satisfied. After truncating the PCE
it will certainly be violated for some θ z , as the expression is now a polynomial in θ(ω), and
polynomials are not bounded. Thus we face the prospect of potential numerical instability, the
computational procedure now does not satisfy Hadamard’s requirements of well-posedness
any more at the discrete level. This was no issue before in section 4.1, as there the operator
was deterministic.
Even if in Eq. (11) the random variables ρk (ω) could be computed exactly and not through
a truncated PCE, there could still be problems, as also the KLE only converges in L2 (R), and
has to be truncated in an actual computation. With more requirements on the covariance
function this convergence may be uniform, and then there would be no problem.
Due to these arguments, we recommend in the case when particular realisations are computed to use the representation Eq. (12) with a pointwise transformation of a Gaussian field,
in our examples of such direct simulations we used this. The representation Eq. (12) makes
16
Hermann G. Matthies
sure that the relation Eq. (21) is satisfied in all cases. We shall see that this difficulty may be
avoided in conjunction with Galerkin projections.
Can we use the Polynomial Chaos Expansion?: In the case of additive noise we started
first by looking at the polynomial chaos expansion of the stochastic noise term, in the spatially
discretised version f (ω). We may do the same here for the coefficient κ(x, ω) =
P (γ)
κ
(x)Hγ (ω), and in the discretised version for the stiffness matrix K(ω). As the stiffγ
ness matrix depends linearly on the conductivity κ(x, ω), one obtains a PCE for the matrix,
similar to Eq. (29):
X
K(ω) =
K (γ) Hγ (ω)
(35)
γ
where of course K (0) = E (K(·)) = K, and each K (γ) is a stiffness matrix computed with
a “conductivity” κ(γ) (x).
Assuming also a PCE of u(ω) as in Eq. (29) when dealing with a stochastic RHS alone,
one has
!
X
X
K (γ) Hγ (ω)
u(α) Hα (ω) = f .
(36)
γ
α
One potential worry—alluded to already previously—is that the PCE in Eq. (35) converges
only in L2 (Ω). We shall deal with this question as soon as we have performed the Galerkin
projection.
To obtain a finite dimensional approximation, truncate the series for u(ω) by letting only
α ∈ JM,p —this subset was defined in relation Eq. (28). Having done that, of course the equation can not be satisfied any more for all ω, and one may use Galerkin projection conditions
to obtain conditions for all β ∈ JM,p :
!
X
X
(37)
E (Hβ (·)Hγ (·)Hα (·)) K (γ) u(α) = E (Hβ (·)f ) = f (β) .
α∈JM,p
γ∈J
Defining new matrices ∆(γ) with elements (∆(γ) )α,β = E (Hβ (·)Hγ (·)Hα (·)), and defining
block vectors u = [. . . , u(α) , . . .] and similarly f , one can write this equation with the Kronecker product as
"
#
X
Ku :=
∆(γ) ⊗ K (γ) u = f .
(38)
γ
The question comes whether, and if so when, one should truncate the PCE of Eq.(35), as
has to be done in an actual computation. It is good to know that this problem is resolved when
looking at the projected equation Eq. (38) (see [62] for a proof): The series in γ in Eqs. (37,
38) is a finite sum, even when the PCE Eq. (35) is not.
To come back to the question in this section’s heading, the answer is yes when we use the
Galerkin method, otherwise, i.e. in a Monte Carlo simulation, not directly or without other
precautions.
The sum and Kronecker product structure allow savings in memory usage and coarse
grain parallelisation in the numerical solution process, and we refer to the next section for
references and more details on this topic.
Unfortunately, often many terms have to be used in the PCE, and we would like to reduce
this. One possibility is shown next.
17
Computational Aspects of Probability
Using the Karhunen-Loève Expansion of the Matrix: As already alluded to, the direct use
of the KLE of the coefficient of conductivity Eq. (11), and subsequent truncation may lead
to numerical problems. One might certainly ask why not use here also the transformation
method Eq. (12) which was advocated before for the Monte Carlo method. The appeal of the
direct KLE is that a similar Kronecker product structure as Eq. (38) with very few terms in
the sum may be achieved, and the computation of these terms is completely analogous to the
Figure 5: Sparsity Pattern of Kronecker Factor ∆.
normal computation of a stiffness matrix. It can be shown [62] that, similarly to the result
on the PCE, the Galerkin projection enables the direct use ofPthe KLE without any further
hypotheses on the covariance function Cκ (x, y) of κ(x, ω) =  ς ξ (ω)κ (x).
When the space discretisation is performed for such an expansion of the conductivity as
in Eq. (11), one may see right away that
K(ω) = K +
∞
X
ς ξ (ω)K  ,
(39)
=1
where K  is computed by using the KL-eigenfunction κ (x) as conductivity instead of κ(x),
and ς and ξ (ω) are the singular values and eigenfunctions of the KLE of κ(x, ω). Note that
this may be usually computed with existing software, all one has to do to supply another
“material”, namely κ (x).
For the discrete u(ω) use again the PCE as before, and impose the Galerkin conditions to
obtain the coefficients:
X
∀α ∈ JM,p :
E (Hα (·)K(·)Hβ (·)) u(β) = E (Hα (·)f ) = f (α) ,
(40)
β∈JM,p
18
Hermann G. Matthies
Expanding Eq. (40) with the series in Eq. (39) gives for all α ∈ JM,p :
"
#
!
"
#
∞
∞
X
X
X
X
()
E Hα (·) K +
ς ξ (·)K  Hβ (·) =
K+
∆α,β K 
β∈JM,p
=1
(41)
=1
β∈JM,p
where
()
∆α,β = ς E (Hα (·)ξ (·)Hβ (·)) .
(42)
P (γ)
To compute such an expectation, again use the PCE of ξ (ω) = γ c Hγ (ω) as in Eq. (14).
From the previous discussion one knows that the PCE series are in this case only finite sums:
P
(γ) (γ)
()
E (Hα (·)ξ (·)Hβ (·)) = γ∈JM,2p c ∆α,β . Define the matrices ∆() with elements ∆α,β from
Eq. (42), and set ∆(0) = I, K 0 = K. Using again the block vectors u = [. . . , u(α) , . . .] and
f , one may write this equation as
"∞
#
X
Ku :=
∆() ⊗ K  u = f .
(43)
=0
One may further expand for j > 0: ∆() =
(γ)
(γ)
γ∈JM,2p ς c ∆ ,
P
such that with Eq. (43)


∞
X
X
(γ)
Ku = 
ς c(γ)
⊗ K  u = f .
 ∆
(44)
=0 γ∈JM,2p
In Fig. 5 we see the sparsity pattern of ∆() , depending on how many terms were used
in the PCE, produced with the MATLAB spy function. White space corresponds to zeros,
whereas each dot represents one full spatial matrix with the size and sparsity pattern of K. A
remark similar as before applies here as well: There is at least one PC-term for each KL-term,
hence we can expect that in an actual computation we have many less terms in Eq. (43) than
in Eq. (38) —although formally there is still an infinite series in Eq. (43).
As noted before, the KLE converges in L2 (R), whereas there is only stability against
perturbations in L∞ (R). We need uniform convergence to be able to truncate the series and
still guarantee the conditions Eq. (21). But the Galerkin projection helps again, as may be
shown (see [62] for a proof): The series in Eq. (43) resp. Eq. (44) converges uniformly. Hence
a finite number of terms suffices to keep the discrete operators K uniformly—in the discretisation of V ⊗ S —positive definite, and therefore their inverses uniformly bounded, assuring
the stability of the approximation process.
The Eq. (43) is again in tensor- or Kronecker product form, and for the computation it
is definitely kept in this way [27, 73, 60, 40, 42, 62, 46]. Solution methods used are usually
of the Krylov subspace type [30], where only multiplication with the system matrix K is
necessary. In our example all the matrices K  and ∆() are symmetric, hence so is K. The
matrices are also positive definite, therefore preconditioned conjugate gradients may be used.
There are again plenty of opportunities for coarse level parallelism, obvious in the sum and
in the Kronecker product, this is described in more detail in [40, 42, 62, 46].
The PCE also gives a natural multi-level structure to the equations, which can be used
in the solution process [61, 62, 46]. An additional possibility is to select the approximating
subspaces adaptively according to the functional which one wants to compute [43, 46]. An
19
Computational Aspects of Probability
Realization of κ
Realization of solution
10
12
15
11
10
10
9
5
2
8
9
8.5
8
8
6
7.5
4
2
7
0
1
9.5
10
7
6.5
0
1
1
1
0 2
6
5.5
0 2
Figure 6: Realisations of Material and Solution on an L-shaped Region.
Variance of solution
Mean of solution
10
10
5
0
2
5
9
6
8
4
7
2
3
6
0
2
2
4
5
0
1
0
1
4
1
1
1
0 2
0 2
Figure 7: Mean and Variance of Solution on an L-shaped Region.
example for a realisation for a coefficient κ(x, ω) and a solution u(x, ω) is shown in Fig. 6.
Since these are random variables at each point, it might be more instructive to consider the
pointwise mean ū(x) and the pointwise variance Cu (x, x), shown in Fig. 7
D IRECT OR N ON -I NTRUSIVE C OMPUTATION : When one wants to compute the PCcoefficients directly, remember that now also the matrix is stochastic. Approximating integrals by a quadrature rule—this could be Monte Carlo—one obtains
u
(α)
≈
Z
X
z=1
wz Hα (θ z )K
−1
(θ z )f (θ z ) =
Z
X
wz Hα (θ z )u(θ z ),
(45)
z=1
with integration weights wz , which in the case of Monte Carlo are simply 1/Z.
Instead of one large system equations Eq. (44), one has to solve many—indeed Z —small
ones K(θ z )u(θ z ) = f (θ z ) for certain realisations θ z (ω). One caveat is again that here one
can not use the PC- and KL-expansion directly to represent the stochastic field, as positive
definiteness and boundedness will be violated, but instead the transformation method Eq. (12)
in section 2.2.
See [44] for a study when it is advisable to use what kind of integration in this direct
computation of PC-coefficients. It is not yet clear when it is better to use the direct approach
just introduced, and when to use the coupled systems of the previous sections.
20
4.3
Hermann G. Matthies
Non-Linear Systems
As often, the ability to solve the linear or at least linearised problem is the “workhorse”
also for non-linear problems. Look back and consider again our model problem Eq. (20).
Assume now that the hydraulic conductivity κ(x, u) depends also on the hydraulic head and
on soil properties which are described by two other fields, κ̂(x) and κ̌(x). We use the model
κ(x, u) = κ̂(x) + κ̌(x)u(x)2 , and we assume that both fields satisfy the boundedness and
positivity conditions Eq. (21). This should be seen as a first approximation to more accurately
modelled non-linear behaviour.
D ISCRETISATION AND N UMERICAL A PPROXIMATION : As before, a standard spatial
discretisation—the same finite element spaces may be used [12, 87, 98] —leads to a problem
of the form
B(ω)[u(ω)] = f (ω),
(46)
where both the non-linear operator B(ω)[·] and the RHS f (ω) are random, and hence so is
u(ω). Once
P we write the solution u(ω) as a linear combination of Hermite polynomials in
u(ω) = α u(α) Hα (ω), and now one has to compute the coefficients u(α) .
Due to the non-linearity, it is of no big advantage to take into detailed account the different sources of randomness, the operator or the RHS, as the superposition principle fails in
general.
C OMPUTATIONAL P ROCEDURES : As often when dealing with a non-linear equation, the
different approaches for linear equations are not really much different any more. A linear
expansion of the right hand side is not carried through to the solution. But certainly it pays to
use the KLE of the random fields involved, if only to deal with as few stochastic dimensions
as possible. For the solution the only thing really left is to use the PCE.
Monte Carlo Methods—Encore: There is not much to add here, as most of the relevant descriptions and warnings were given before. Generate samples θ z , solve the equations etc., as
described already. But now these are non-linear equations, so the system has to be solved—
although independently and possibly in parallel—many times. This can be very costly, depending on what kind of integration procedure was used [44].
Using the Polynomial Chaos Expansion: The Galerkin method is obtained by inserting
the stochastic ansatz for the PCE of u(ω) into Eq. (46). In general there will be a residuum
R(ω)[u(ω)] = f (ω) − B(ω)[u(ω)],
(47)
which is then projected in a standard Galerkin manner onto the finite dimensional stochastic
subspace span{Hα |α ∈ JM,p }, and one requires the projection to vanish. This results in
r(u) = [. . . , E (Hα (·)R(·)[uH(·)]) , . . .] = 0,
(48)
where the same block vectors— u = [. . . , u(α) , . . .] —as before are used.
Now Eq. (48) is a huge non-linear system, and one way to approach it is through the use
of Newton’s method, which involves linearisation and subsequent solution of the linearised
system, employing the methods of the preceeding sections 4.1 and 4.2.
Another possibility, avoiding the costly linearisation and solution of a new linear system
at each iteration, is the use of Quasi-Newton methods [57, 16]. This was done in [44], and the
Quasi-Newton method used—as we have a symmetric positive definite or potential minimisation problem this was the BFGS-update—performed very well. The Quasi-Newton methods
21
Computational Aspects of Probability
Solution Mean
Solution Standard Deviation
4
0.4
3
0.3
2
0.2
−1
1
0
1
0
−0.5
x
0
0.5
0
0.5
−1 1
−0.5
0
1
0
0.5
−1
0.1
−0.5
−0.5
y
x
0.5
−1 1
y
Figure 8: Mean and Standard Deviation of Solution to Non-linear Model.
produce updates to the inverse of a matrix, and these low-rank changes [16] are also best kept
in tensor product form [57]; so that we have tensor products here on two levels, which makes
for a very economical representation.
But in any case, in each iteration the residual Eq. (48) has to be evaluated at least once,
which means that for all α ∈ JM,p the integral
Z
E (Hα (·)R(·)[uH(·)]) =
Hα (ω)R(ω)[uH(ω)] dP (ω)
Ω
has to be computed. In general this can not be done analytically as before in the case of linear
equations, and one has to resort to numerical quadrature rules:
Z
Hα (ω)R(ω)[uH(ω)] dP (ω) ≈
Ω
Z
X
wz Hα (θ z )R(θ z )[u(θ z )].
z=1
What this means is that for each evaluation of the residual Eq. (48) the spatial residual Eq. (47)
has to be evaluated Z times—once for each θ z where one has to compute R(θ z )[u(θ z )].
Certainly this can be done independently and in parallel without any communication. But we
would like to point out that instead of solving the system every time for each θ z as in the
preceding section, here we only have to compute the residual—but this for every iteration.
This emphasis on integration now also points towards the direct or non-intrusive methods
already mentioned earlier.
We compute the solution for the non-linear groundwater flow model. The soil parameter
κ(x, ω) is chosen beta-distributed as indicated before. As a reference, the mean and standard
deviation were computed by Smolyak quadrature S66 —to be explained in section 4.4 —in
altogether Z = 6, 188 integration points. They are show in Fig. 8.
Next we compute the PC-expansion via the Galerkin method explained in this section, the
error of which for the mean is shown in Fig. 9. We choose a polynomial chaos of degree 2 in
6 independent Gaussian variables as ansatz (28 stochastic functions). A spatial discretisation
in 170 degrees of freedom was performed, totalling 4, 760 non-linear equations. The BFGS
22
Hermann G. Matthies
Error ·104 in Mean for Galerkin
P rob{u > 3.25}
1
0.8
0.8
0.6
0.6
0.4
0.4
−1
−0.8
−0.6
−0.4
−0.2
0.2
0
1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1 1
x
0
0.2
0.4
0.6
0.8
y
−1
0.2
−0.5
0
1
0
0.5
0
−0.5
x
0.5
−1 1
y
Figure 9: Error in Mean for PC-Galerkin and Example Statistic.
u(α) for Galerkin PC-Expansion
0.08
0.06
0.04
−1
−0.8
−0.6
−0.4
−0.2
0.02
0
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0
−0.2
0.6
−0.4
0.8
−0.6
y
−0.8
−1 1
x
Error ·104 in u(α)
3
2.5
2
1.5
1
0.5
0
−0.5
1
0.8
−1
−0.8
−0.6
−0.4
−0.2
0.6
0.4
0.2
0
0.2
0.4
0
−0.2
0.6
−0.4
0.8
−0.6
y
−0.8
−1 1
x
Figure 10: PC-Galerkin Vector and its Error.
solver required 19 iterations, and as the first iterations required line-searches, the residual
had to be evaluated 24 times. The residual was integrated by the 5-stage Smolyak quadrature
S56 in Z = 1, 820 integration points. As the evaluation in each integration point requires one
integration in the spatial dimension, 43, 680 spatial integrations were performed.
As we now have a “response surface”, we show also in Fig. 9 as an example the computation of pu0 (x) = Pr{u(x) > 3.25} for all points in the domain R. Next we show one of the
PC-vectors in Fig. 10, and its error. It is small, at least in the “eye-ball” norm.
Direct or Non-Intrusive Computation—Finale: As before, recall that the PC-coefficients
of u(ω) are also functionals—u(α) = E (Hα (·)u(·)). When trying to evaluate this numerically, one has to solve the non-linear system for each integration point θ z ; this is very similar
to the MC method. Only that we multiply by the corresponding polynomial Hα before adding
23
Computational Aspects of Probability
everything up.
In principle this integral could be computed via MC integration, but now a considerable
weakness of MC and QMC methods comes to the surface—they can not exploit the smoothness of the integrand. The integrand is in our case partly a polynomial and hence very smooth,
but has very high variance—and that is where MC methods have problems [44].
We therefore focus finally on high-dimensional integration, a subject which was popping
up already in several places.
4.4
High-Dimensional Integration
We only want to sketch the various possibilities, and draw some comparisons. For more in
depth information we refer to the references.
Suppose that we want to evaluate the integral Eq. (49) by approximating it with a sum—a
quadrature formula.
Z
E (Ψ(·)) ≈ E (Ψ(·))m =
Ψ(ω) dP (ω) ≈ ΨZ :=
Θm
Z
X
wz Ψ(θ z ),
(49)
z=1
where now m is the number of independent random variables, the dimension of the integration domain Θm . The function Ψ(ω) is a substitute for the different integrands we have
encountered so far, e.g. the integrand Ψ(x, ω, u(x, ω)) from Eq. (17), whose numerical approximation was sketched in Eq. (18). Several possibilities exist, and we briefly look at some
of them.
M ONTE C ARLO M ETHODS : Monte Carlo methods (MC methods)—already described in
sections 3.1 and 4.1 —obtain the integration points as Z independent random realisations of
ω z ∈ Ωm distributed according to the probability-measure Γ on Ωm , and use constant weights
wz = 1/Z. MC methods are probabilistic as the integration points are chosen randomly, and
therefore the approximation and the error are random variables. For large Z, the error is approximately kΨ̃kL2 Z −1/2 N (0, 1), where N (0, 1) is a standard-distributed Gaussian random
variable, and the L2 (Ω)-norm is the standard deviation of the zero mean fluctuating part Ψ̃ of
the integrand.
Due to the O(kΨ̃kL2 Z −1/2 ) behaviour of the error, MC methods converge slowly—for
instance, the error is reduced by one order of magnitude if the number of evaluations is increased by two orders. The MC methods are well suited for integrands with small variance
and low accuracy requirements. In applications, their efficiency is usually increased somewhat by variance reduction and importance sampling, see e.g. [10, 79, 80] and the references
therein. The significant advantage of MC methods is that their convergence rate is independent of the dimension, in contrast with the other methods to be discussed.
Q UASI -M ONTE C ARLO M ETHODS : Quasi-Monte Carlo methods (QMC) are often seen
as an alternative to Monte Carlo methods, e.g. [66, 10]. Informally speaking, they choose
the sequence of integration points such that “for any number of points Z the integral E (1)
is approximated well by the sequence”. Such sequences are called quasi-random numbers or
low discrepancy sequences [66].
The most commonly used QMC methods have an error of O kΨ̃kBV Z −1 (log Z)m ,
where kΨ̃kBV denotes the bounded variation norm. If the dimension is not too large and
24
Hermann G. Matthies
the integrand is smooth, the term Z −1 dominates the error and QMC methods may be more
efficient than MC methods, e.g. see [10] and the references therein.
N ORMAL Q UADRATURE RULES : The textbook approach to an integral like Eq. (49)
would be to take a good one-dimensional quadrature rule, and to iterate it in every dimension;
this we might call the full tensor product approach.
Assume that we use one-dimensional Gauss-Hermite-formulas Qk with k ∈ N integration
points ω ,k and weights w,k ,  = 1, . . . , k. As is well-known, they integrate polynomials of
degree less than 2k exactly, and yield an error of order O(k −(2r−1) ) for r-times continuously
differentiable integrands, hence takes smoothness into full account.
If we take a tensor product of these rules by iterating them m times, we have
ΨZ =
Qm
k (Ψ)
:= (Qk ⊗ · · · ⊗ Qk )(Ψ) =
m
O
Qk (Ψ)
=1
=
k
X
1 =1
···
Z
X
w1 ,k · · · wm ,k Ψ(ω 1 ,k , . . . , ω m ,k ).
m =1
This “full” tensor quadrature evaluates the integrand on a regular mesh of Z = k m points,
and the approximation-error has order O(Z −(2r−1)/m ). Due to the exponential growth of the
number of evaluation points and hence the effort with increasing dimension, the application
of full tensor quadrature is impractical for high stochastic dimensions, this has been termed
the “curse of dimension” [67].
S PARSE OR S MOLYAK Q UADRATURE : “Sparse” or Smolyak quadrature [85] can be
applied in much higher dimensions—for recent work see e.g. [67, 68, 75] and the references
therein. A software package is available at [74].
Like full tensor quadrature, a Smolyak quadrature formula is constructed from tensor
products of one-dimensional quadrature formulas, but it combines quadrature formulas of
high order in only some dimensions with formulas of lower order in the other dimensions.
For a multi-index η ∈ Nm with |η| as before, the Smolyak quadrature formula is
m
X
k−1 O
m
k+m−1−|η|
ΨZ = Sk (Ψ) :=
(−1)
Qη (Ψ).
|η| − k
k≤|η|≤k+m−1
=1
For a fixed k the number of evaluations grows significantly slower in the number of
dimensions than for full quadrature. The price is a larger error: full quadrature integrates
ηm
exactly if their partial degree max η does not exceed 2k − 1.
monomials θ η = θ1η1 · · · θm
m
Smolyak formulas Sk integrate multivariate polynomials exactly only if their total polynomial degree
2k − 1. But still the error is only O(Z −k (log Z)(m−1)(k+1) ) with
k |η| isat most
Z = O 2 /(k!) mk evaluation points. This only grows polynomially in the dimension,
and has been used up to several hundred dimensions.
N UMERICAL E XPERIMENTS : We used Smolyak quadrature S46 with 451 integration
points and Monte Carlo simulation with 500 integration points. The accuracy in mean and
standard deviation with respect to the reference solution are shown in Fig. 11. The errors from
the naive Monte Carlo simulation are considerably larger than the error from the Smolyak
integration—about forty times larger for the mean and six times larger for the standard deviation. Thus, a naive Monte Carlo simulation would require an approximately 1, 600 times
higher effort to obtain the same accuracy.
25
Computational Aspects of Probability
Error ·104 in Mean for S46
(Z = 455 integration points).
Error in Mean for Monte Carlo (Z = 500).
6
0.02
0.015
4
0.01
2
−1
−0.5
0
1
0
−0.5
x
−0.5
0
1
0
0.5
−1
0.005
0
0.5
0
0.5
−0.5
y
−1 1
x
Error ·103 in Std.-Dev. for S46 (Z = 455).
y
−1 1
Error in Std-Dev. for Monte Carlo
(Z = 500).
4
0.02
3
0.015
2
0.5
0.01
−1
1
−0.5
0
1
0
0.5
0
−0.5
x
−0.5
0
1
0
0.5
0
0.5
−1 1
−1
0.005
y
−0.5
x
Figure 11: Solution Errors by Direct Computation
0.5
−1 1
y
26
Hermann G. Matthies
See [44] for some experiments, when to use which kind of integration. The finding there
is that for low m normal quadrature is best. For higher to moderately high (several hundred)
m, sparse or Smolyak quadrature [85, 23, 75, 74] is advisable. For very high dimensions, we
come into the region where first Quasi Monte Carlo [66] and then finally for extremely high
dimension Monte Carlo methods should be most effective.
5
Conclusion and Outlook
This lecture has tried to illuminate some aspects of current computational probabilistic methods. Here the idea of random variables as functions in an infinite dimensional space which
have to be approximated by elements of finite dimensional spaces has brought a new view to
the field. It allows the Galerkin methods, which have been so useful in the approximation of
partial differential and integral equations, to be applied to this case as well. With this comes
the whole range of techniques developed in that area for fast and efficient solution, error estimation, adaptivity, etc. On a more philosophical level, a stochastic problem is in that way
converted to a large deterministic one.
It has been attempted to contrast these new methods with the more traditional Monte Carlo
approaches. For the new Galerkin methods, the polynomial chaos expansion—or something
equivalent—seems to be fundamental. The Karhunen-Loève expansion can in that light rather
be seen as a model reduction strategy. Its full use as a data reduction device is yet to be
explored. The possibility of computing the coefficients of the stochastic expansion both as
the solution of a large coupled system, and as evaluations of certain integrals of the response
brings the circle back to the well-known Monte Carlo methods when seen as an integration
rule. It remains yet to be seen which of these approaches is more effective in which situation.
References
[1] R. J. Adler: The Geometry of Random Fields. John Wiley & Sons, Chichester, 1981.
[2] K. E. Atkinson: The Numerical Solution of Integral Equations of the Second Kind. Cambridge University
Press, Cambridge, 1997.
[3] G. Augusti, A. Baratta and F. Casciati: Probabilistic Methods in Structural Engineering. Chapman and
Hall, London, 1984.
[4] I. Babuška and P. Chatzipantelidis: On solving elliptic stochastic partial differential equations, Comp.
Meth. Appl. Mech. Engrg. 191 (2002) 4093–4122.
[5] J. S. Bendat and A. G. Piersol: Engineering Applications of Correlation and Spectral Analysis. John
Wiley & Sons, Chichester, 1980.
[6] F. E. Benth and J. Gjerde: Convergence rates for finite element approximations for stochastic partial
differential equations. Stochastics and Stochochastics Reports 63 (1998) 313–326.
[7] A. V. Balakrishnan: Stochastic Differential Systems. Lecture Notes in Economics and Mathematical
Systems 84. Springer-Verlag, Berlin, 1973.
[8] P. Besold: Solutions to Stochastic Partial Differential Equations as Elements of Tensor Product Spaces.
Doctoral thesis, Georg-August-Universität, Göttingen, 2000.
[9] R. N. Bracewell: The Fourier Transform and Its Applications. McGraw-Hill, New York, NY, 1978.
[10] R. E. Caflisch: Monte Carlo and quasi-Monte Carlo methods. Acta Numerica 7 (1998) 1–49.
[11] G. Christakos: Random Field Models in Earth Sciences. Academic Press, San Diego, CA, 1992.
Computational Aspects of Probability
27
[12] P. G. Ciarlet: The Finite Element Method for Elliptic Problems. North-Holland, Amsterdam, 1978.
[13] R. W. Clough and J. Penzien: Dynamics of Structures. McGraw-Hill, New York, NY, 1975.
[14] R. Courant and D. Hilbert: Methods of Mathematical Physics. John Wiley & Sons, Chichester, 1989.
[15] M. K. Deb, I. Babuška, and J. T. Oden: Solution of stochastic partial differential equations using Galerkin
finite element techniques. Comp. Meth. Appl. Mech. Engrg. 190 (2001) 6359–6372.
[16] J. E. Dennis, Jr. and R. B. Schnabel: Numerical Methods for Unconstrained Optimization and Nonlinear
Equations. SIAM, Philadelphia, PA, 1996.
[17] A. Der Kuireghian and J.-B. Ke: The stochastic finite element method in structural reliability. J. Engrg.
Mech. 117 (1988) 83–91.
[18] I. Doltsinis: Inelastic deformation processes with random parameters—methods of analysis and design.
Comp. Meth. Appl. Mech. Engrg. 192 (2003) 2405–2423.
[19] J. L. Doob: Stochastic Processes. John Wiley & Sons, Chichester, 1953.
[20] I. Elishakoff: Are probabilistic and antioptimization methods interrelated? In [21].
[21] I. Elishakoff (ed.): Whys and Hows in Uncertainty Modelling—Probability, Fuzziness, and AntiOptimization. Springer-Verlag, Berlin, 1999.
[22] C. W. Gardiner: Handbook of Stochastic Methods. Springer-Verlag, Berlin, 1985.
[23] T. Gerstner and M. Griebel: Numerical integration using sparse grids. Numer. Algorithms 18 (1998)
209–232.
[24] R. Ghanem: Ingredients for a general purpose stochastic finite elements implementation. Comp. Meth.
Appl. Mech. Engrg. 168 (1999) 19–34.
[25] R. Ghanem: Stochastic finite elements for heterogeneous media with multiple random non-Gaussian
properties. ASCE J. Engrg. Mech. 125 (1999) 24–40.
[26] R. Ghanem and P. D. Spanos: Stochastic Finite Elements—A Spectral Approach. Springer-Verlag, Berlin,
1991.
[27] R. Ghanem and R. Kruger: Numerical solution of spectral stochastic finite element systems. Comp. Meth.
Appl. Mech. Engrg. 129 (1999) 289–303.
[28] I. I. Gihman and A. V. Skorokhod: Stochastic Differential Equations. Springer-Verlag, Berlin, 1972.
[29] I. I. Gihman and A. V. Skorokhod: Introduction to the Theory of Random Processes. Dover Publications,
New York, NY, 1996.
[30] G. Golub and C. F. Van Loan: Matrix Computations. Johns Hopkins University Press, Baltimore, MD,
1996.
[31] M. Grigoriu: Applied non-Gaussian Processes. Prentice Hall, Englewood Cliffs, NJ, 1995.
[32] M. Grigoriu: Stochastic Calculus—Applications in Science and Engineering. Birkhäuser Verlag, Basel,
2002.
[33] A. Haldar and S. Mahadevan: Reliability assessment using stochastic finite element analysis. John Wiley
& Sons, Chichester, 2000.
[34] H. Holden, B. Øksendal, J. Ubøe, and T.-S. Zhang: Stochastic Partial Differential Equations. Birkhäuser
Verlag, Basel, 1996.
[35] S. Janson: Gaussian Hilbert Spaces. Cambridge University Press, Cambridge, 1997.
[36] H. Jeggle: Nichtlineare Funktionalanalysis. BG Teubner, Stuttgart, 1979.
[37] M. Jardak, C.-H. Su, and G. E. Karniadakis: Spectral polynomial chaos solutions of the stochastic advection equation. SIAM J. Sci. Comput. 17 (2002) 319–338.
[38] G. Kallianpur: Stochastic Filtering Theory. Springer-Verlag, Berlin, 1980.
28
Hermann G. Matthies
[39] A. Keese: A review of recent developments in the numerical solution of stochastic PDEs (stochastic finite
elements). Technical report, Informatikbericht 2003-6, Institute of Scientific Computing, Department of
Mathematics and Computer Science, Technische Universität Braunschweig, Brunswick, 2003.
http://opus.tu-bs.de/opus/volltexte/2003/504/
[40] A. Keese and H. G. Matthies: Parallel solution of stochastic PDEs. Proc. Appl. Math. Mech. 2 (2003)
485–486.
[41] A. Keese and H. G. Matthies: Efficient solvers for nonlinear stochastic problems. In Proceedings of
WCCM V—5th World Congress Comput. Mech., Vienna, (7–12 July 2002).
http://wccm.tuwien.ac.at/
[42] A. Keese and H. G. Matthies: Parallel solution methods for stochastic systems. pp. 2023–2025 in K.J. Bathe (ed.), Computational Fluid and Solid Mechanics—Proceedings of the 2nd MIT conference. Elsevier, Amsterdam, 2003.
[43] A. Keese and H. G. Matthies: Adaptivity and sensitivity for stochastic problems. pp. 311–316 in
P. D. Spanos and G. Deodatis (eds.), Computational Stochastic Mechanics 4. Millpress, 2003.
[44] A. Keese and H. G. Matthies: Numerical methods and Smolyak quadrature for nonlinear stochastic partial differential equations. Technical report, Informatikbericht 2003-5, Institute of Scientific Computing,
Department of Mathematics and Computer Science, Technische Universität Braunschweig, Brunswick,
2003.
http://opus.tu-bs.de/opus/volltexte/2003/471/
[45] A. Keese and H. G. Matthies: Parallel Computation of Stochastic Groundwater Flow. Technical report,
Informatikbericht 2003-9, Institute of Scientific Computing, Department of Mathematics and Computer
Science, Technische Universität Braunschweig, Brunswick, 2003.
http://opus.tu-bs.de/opus/volltexte/2003/505/
[46] A. Keese: Numerical Solution of Systems with Stochastic Uncertainties—A General Purpose Framework
for Stochastic Finite Elements. Doctoral thesis, Technische Universität Braunschweig, Brunswick, 2003.
[47] M. Kleiber and T. D. Hien: The Stochastic Finite Element Method. Basic Perturbation Technique and
Computer Implementation. John Wiley & Sons, Chichester, 1992.
[48] P. E. Kloeden and E. Platen: Numerical Solution of Stochastic Differential Equations. Springer-Verlag,
Berlin, 1995.
[49] A. Khuri and J. Cornell: Response Surfaces: Designs and Analyses. Marcel Dekker, New York, NY, 1987.
[50] P. Krée and C. Soize: Mathematics of Random Phenomena. D. Reidel, Dordrecht, 1986.
[51] W.-K. Liu, T. Belytschko, and A. Mani: Probabilistic finite elements for nonlinear structural dynamics.
Comp. Meth. Appl. Mech. Engrg. 56 (1986) 61–86.
[52] M. Loève: Probability Theory. Springer-Verlag, Berlin, 1977.
[53] L. D Lutes: A perspective on state-space stochastic analysis. Proc. 8th ASCE Speciality Conf. Prob. Mech.
Struct. Reliability, 2000.
[54] H. O. Madsen, S. Krenk and N.C. Lind: Methods of Structural Safety. Prentice Hall, Englewood Cliffs,
NJ, 1986.
[55] G. Maglaras, E. Nikolaidis, R. T. Hafka and H. H. Cudney: Analytical-experimental comparison of probabilistic and fuzzy set methods for designing under uncertainty. In [64].
[56] P. Malliavin: Stochastic Analysis. Springer-Verlag, Berlin, 1997.
[57] H. Matthies and G. Strang: The solution of nonlinear finite element equations. Int. J. Numer. Meth. Engrg.
14 (1979) 1613–1626.
[58] H. G. Matthies, C. E. Brenner, C. G. Bucher, and C. Guedes Soares: Uncertainties in probabilistic numerical analysis of structures and solids—stochastic finite elements. Struct. Safety 19 (1997) 283–336.
[59] H. G. Matthies and C. G. Bucher: Finite elements for stochastic media problems. Comp. Meth. Appl.
Mech. Engrg. 168 (1999) 3–17.
Computational Aspects of Probability
29
[60] H. G. Matthies and A. Keese: Fast solvers for the white noise analysis of stochastic systems. Proc. Appl.
Math. Mech. 1 (2002) 456–457.
[61] H. G. Matthies and A. Keese: Multilevel solvers for the analysis of stochastic systems. pp. 1620–1622
in K.-J. Bathe (ed.), Computational Fluid and Solid Mechanics—Proceedings of the 1st MIT conference.
Elsevier, Amsterdam, 2001.
[62] H. G. Matthies and A. Keese: Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations. Technical report, Informatikbericht 2003-8, Institute of Scientific Computing, Department
of Mathematics and Computer Science, Technische Universität Braunschweig, Brunswick, 2003.
http://opus.tu-bs.de/opus/volltexte/2003/489/
submitted to Comp. Meth. Appl. Mech. Engrg..
[63] R. E. Melchers: Structural Reliability Analysis and Prediction. John Wiley & Sons, Chichester, 1999.
[64] H. G. Natke and Y. Ben-Haim (eds.): Uncertainty: Models and Measures. Akademie-Verlag, Berlin, 1997.
[65] A. Naess and S. Krenk (eds.): IUTAM Symposium on Advances in Nonlinear Stochastic Mechanics. Series
S OLID M ECHANICS AND ITS A PPLICATIONS : Vol. 47, Kluwer, Dordrecht, 1996.
[66] H. Niederreiter: Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia, PA,
1992.
[67] E. Novak and K. Ritter: The curse of dimension and a universal method for numerical integration. pp.
177–188 in G. Nürnberger, J. W. Schmidt, and G. Walz (eds.), Multivariate Approximation and Splines,
ISNM. Birkhäuser Verlag, Basel, 1997.
[68] E. Novak and K. Ritter: Simple cubature formulas with high polynomial exactness. Constr. Approx. 15
(1999) 499–522.
[69] J. T. Oden: Qualitative Methods in Nonlinear Mechanics. Prentice-Hall, Englewood Cliffs, NJ, 1986.
[70] B. Øksendal: Stochastic Differential Equations. Springer-Verlag, Berlin, 1998.
[71] M. Papadrakakis and V. Papadopoulos: Robust and efficient methods for stochastic finite element analysis
using Monte Carlo simulation. Comp. Meth. Appl. Mech. Engrg. 134 (1996) 325–340.
[72] A. Papoulis: Probability, Random Variables, and Stochastic Processes. McGraw-Hill, New York, NY,
1984.
[73] M. Pellissetti and R. Ghanem: Iterative solution of systems of linear equations arising in the context of
stochastic finite elements. Adv. Engrg. Softw. 31 (2000) 607–616.
[74] K. Petras: Smolpack—a software for Smolyak quadrature with delayed Clenshaw-Curtis basis-sequence.
http://www-public.tu-bs.de:8080/˜petras/software.html.
[75] K. Petras: Fast calculation of coefficients in the Smolyak algorithm. Numer. Algorithms 26 (2001) 93–109.
[76] F. Riesz and B. Sz.-Nagy: Functional Analysis. Dover Publications, New York, NY, 1990.
[77] K. K. Sabelfeld: Monte Carlo Methods in Boundary Value Problems Springer-Verlag, Berlin, 1991.
[78] S. Sakamoto and R. Ghanem: Simulation of multi-dimensional non-Gaussian non-stationary random fields
Prob. Engrg. Mech. 17 (2002) 167–176.
[79] G. I. Schuëller: A state-of-the-art report on computational stochastic mechanics. Prob. Engrg. Mech. 14
(1997) 197–321.
[80] G. I. Schuëller: Recent developments in structural computational stochastic mechanics. pp. 281–310 in
B.H.V. Topping (ed.), Computational Mechanics for the Twenty-First Century. Saxe-Coburg Publications,
Edinburgh, 2000.
[81] G. I. Schuëller and P. D. Spanos (eds.): Monte Carlo Simulation. Balkema, Rotterdam, 2001.
[82] C. Schwab and R.-A. Todor: Sparse finite elements for stochastic elliptic problems—higher order moments. Computing 71 (2003) 43–63.
30
Hermann G. Matthies
[83] M. Shinozuka and G. Deodatis: Simulation of stochastic processes and fields. Prob. Engrg. Mech. 14
(1997) 203–207.
[84] A. V. Skorokhod: Studies in the Theory of Random Processes. Dover, New York, NY, 1985.
[85] S. A. Smolyak: Quadrature and interpolation formulas for tensor products of certain classes of functions.
Sov. Math. Dokl. 4 (1963) 240–243.
[86] G. Strang: Introduction to Applied Mathematics. Wellesley-Cambridge Press, Wellesley, MA, 1986.
[87] G. Strang and G. J. Fix: An Analysis of the Finite Element Method. Wellesley-Cambridge Press, Wellesley,
MA, 1988.
[88] B. Sudret and A. Der Kiureghian: Stochastic finite element methods and reliability. A state-of-the-art
report. Technical Report UCB/SEMM-2000/08, Department of Civil & Environmental Engineering, University of California, Berkeley, CA, 2000.
[89] T. G. Theting: Solving Wick-stochastic boundary value problems using a finite element method. Stochastics and Stochastics Reports 70 (2000) 241–270.
[90] G. Våge: Variational methods for PDEs applied to stochastic partial differential equations. Math. Scand.
82 (1998) 113–137
[91] E. Vanmarcke, M. Shinozuka, S. Nakagiri and G. I. Schuëller: Random fields and stochastic finite elements. Struct. Safety 3 (1986) 143–166.
[92] E. Vanmarcke: Random Fields: Analysis and Synthesis. The MIT Press, Cambridge, MA, 1988.
[93] N. Wiener: The homogeneous chaos. Amer. J. Math. 60 (1938) 897–936.
[94] D. Xiu and G. E. Karniadakis: The Wiener-Askey polynomial chaos for stochastic differential equations.
SIAM J. Sci. Comput. 24 (2002) 619–644.
[95] D. Xiu, D. Lucor, C.-H. Su, and G. E. Karniadakis: Stochastic modeling of flow-structure interactions
using generalized polynomial chaos. ASME J. Fluid Engrg. 124 (2002) 51–69.
[96] D. Xiu and G. E. Karniadakis: Modeling uncertainty in steady state diffusion problems via generalized
polynomial chaos. Comp. Meth. Appl. Mech. Engrg. 191 (2002) 4927–4948.
[97] X. F. Xu and L. Graham-Brady: Nonlinear analysis of heterogeneous materials with GMC technique.
Proc. 16th ASCE Engrg. Mech. Conf., Seattle, WA, 2003.
[98] O. C. Zienkiewicz and R. L. Taylor: The Finite Element Method. Butterwort-Heinemann, Oxford, 5th ed.,
2000.