Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos Expansion for Uncertainties
Quantification and Sensitivity Analysis
Thierry Crestaux1 , Jean-Marc Martinez1 ,
Olivier Le Maitre2 , Olivier Lafitte 1,3
1
Commissariat à l’Énergie Atomique, Centre d’Études de Saclay, France ;
2
Université d’Évry, France ; 3 Université Paris Nord, France
1/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Introduction
Uncertainties quantification in numerical simulation by Polynomial
Chaos expension is a technic which has been used recently for
numerous problems.
This method can also be used in global sensitivity analysis by the
approximation of sensitivity indices.
2/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Plan
1
Polynomial Chaos expansion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Least square approximation
Non Intrusive Spectral Projection
2
Uncertainty and sensitivity analysis by PC
Uncertainty analysis
Sensitivity Analysis
Sobol decomposition of the PC surrogate model
Sensitivity indices
Examples
3
Application : Advection-dispersion
3/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Polynomial Chaos expansion
4/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Polynomial Chaos
Polynomial Chaos (PC) expansions of (2nd order) stochastic processes :
y(x, t, θ) =
∞
X
βk (x, t)Ψk (ξ(θ)) (Wiener 1938).
k =0
Application to uncertainty quantification by Ghanem and Spanos.
ξ = (ξ1 , ξ2 , . . . , ξd ) a set of d independent
Qsecond order random
variables with given joint density p(ξ) = pi (ξi ).
(Ψk (ξ))k ∈N multidimensional orthogonal polynomials with regard
to the inner product
(mathematical expectation)
R
< Ψk , Ψl >≡ Ψk (ξ)Ψl (ξ)p(ξ)dξ = δkl ||Ψk ||2 .
5/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Polynomial Chaos
Polynomial Chaos (PC) expansions of (2nd order) stochastic processes :
y(x, t, θ) =
∞
X
βk (x, t)Ψk (ξ(θ)) (Wiener 1938).
k =0
Application to uncertainty quantification by Ghanem and Spanos.
ξ = (ξ1 , ξ2 , . . . , ξd ) a set of d independent
Qsecond order random
variables with given joint density p(ξ) = pi (ξi ).
(Ψk (ξ))k ∈N multidimensional orthogonal polynomials with regard
to the inner product
(mathematical expectation)
R
< Ψk , Ψl >≡ Ψk (ξ)Ψl (ξ)p(ξ)dξ = δkl ||Ψk ||2 .
5/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Polynomial Chaos
y(x, t, ξ) =
∞
X
βk (x, t)Ψk (ξ),
k =0
where βk (x, t) are the PC coefficients or stochastic modes of y.
Knowledge of the βk fully characterizes the process y.
For practical use, truncature at polynomial order no :
P +1=
(d + no)!
d!no!
⇒ y(x, t, ξ) ≈
P
X
βk (x, t)Ψk (ξ).
k =0
Fast increase of the basis dimension P according to no.
Need for numerical procedure to compute βk .
Thierry Crestaux
SAMO, JUNE 2007
6/23
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Polynomial Chaos
y(x, t, ξ) =
∞
X
βk (x, t)Ψk (ξ),
k =0
where βk (x, t) are the PC coefficients or stochastic modes of y.
Knowledge of the βk fully characterizes the process y.
For practical use, truncature at polynomial order no :
P +1=
(d + no)!
d!no!
⇒ y(x, t, ξ) ≈
P
X
βk (x, t)Ψk (ξ).
k =0
Fast increase of the basis dimension P according to no.
Need for numerical procedure to compute βk .
Thierry Crestaux
SAMO, JUNE 2007
6/23
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Intrusive method : Galerkin projection
Galerkin projection
A two steps procedure to solve spectral problems :
The introduction of the truncated spectral expansions into model
equations.
Determination of the PC coefficients such that the residual is
orthogonal to the basis.
*
+
X
M(y; D(θ)) = 0 ⇒ M(
βi Ψi (ξ(θ)); D(θ)), Ψk (ξ(θ)) = 0 ∀k.
i
Comments :
? A set of P + 1 coupled spectral problems.
? Require rewriting / adaptation of existing codes.
7/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Intrusive method : Galerkin projection
Galerkin projection
A two steps procedure to solve spectral problems :
The introduction of the truncated spectral expansions into model
equations.
Determination of the PC coefficients such that the residual is
orthogonal to the basis.
*
+
X
M(y; D(θ)) = 0 ⇒ M(
βi Ψi (ξ(θ)); D(θ)), Ψk (ξ(θ)) = 0 ∀k.
i
Comments :
? A set of P + 1 coupled spectral problems.
? Require rewriting / adaptation of existing codes.
7/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Non-intrusive methods
Construction of a sample set {ξ (i) } of ξ and corresponding set of
deterministic solutions {y (i) = y(x, t, ξ(i) )}.
Use the solution set to estimate/compute the PC coefficients β k .
Comments :
⊕ Solve a (large) number of deterministic problems.
⊕ Transparent to non linearities.
Convergence with the sample set dimension and error estimation.
Currently we use two different non-intrusive methods :
Least square approximation of the βk .
Non Intrusive Spectral Projection (NISP).
8/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Non-intrusive methods
Construction of a sample set {ξ (i) } of ξ and corresponding set of
deterministic solutions {y (i) = y(x, t, ξ(i) )}.
Use the solution set to estimate/compute the PC coefficients β k .
Comments :
⊕ Solve a (large) number of deterministic problems.
⊕ Transparent to non linearities.
Convergence with the sample set dimension and error estimation.
Currently we use two different non-intrusive methods :
Least square approximation of the βk .
Non Intrusive Spectral Projection (NISP).
8/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Least square approximation
Least square problem for a sample sets B = (ξ (i) ) and y = (y (i) ).
β̂ R (B) = (Z T Z )−1 Z T y
where Z T Z is the Fisher matrix :

1 Ψ1 (ξ(1) )
 1 Ψ1 (ξ(2) )

Z = .
..
 ..
.
1
Ψ1 (ξ(n) )
. . . ΨP (ξ(1) )
. . . ΨP (ξ(2) )
..
..
.
.
. . . ΨP (ξ(n) )
Open questions :
Selection of the sample set ?
Design Optimal Experiment, active learning ?
Error estimation ?
Model selection ?
Thierry Crestaux
SAMO, JUNE 2007





9/23
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Least square approximation
Least square problem for a sample sets B = (ξ (i) ) and y = (y (i) ).
β̂ R (B) = (Z T Z )−1 Z T y
where Z T Z is the Fisher matrix :

1 Ψ1 (ξ(1) )
 1 Ψ1 (ξ(2) )

Z = .
..
 ..
.
1
Ψ1 (ξ(n) )
. . . ΨP (ξ(1) )
. . . ΨP (ξ(2) )
..
..
.
.
. . . ΨP (ξ(n) )
Open questions :
Selection of the sample set ?
Design Optimal Experiment, active learning ?
Error estimation ?
Model selection ?
Thierry Crestaux
SAMO, JUNE 2007





9/23
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Non Intrusive Spectral Projection : NISP
Exploit orthogonality of the PC basis :
βk =
hy(ξ), Ψk (ξ)i
2
,
Ψk
hy(ξ), Ψk i =
Z
y(ξ)Ψk (ξ)pdf (ξ)dξ.
Ω
Numerical integration :
Z
Ω
y(ξ)Ψk (ξ)pdf (ξ)dξ ≈
N
X
i=1
y(ξ (i) )Ψk (ξ(i) )w (i) = β̂k Ψ2k ,
with ξ (i) and w (i) are integration quadrature points / weights.
⊕ Independent computation of the PC coefficients.
Curse of dimension (cubature formula, adpative construction,
Monte-Carlo, . . .)
10/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Polynomial Chaos
Intrusive method : Galerkin projection
Non-intrusive methods
Non Intrusive Spectral Projection : NISP
Exploit orthogonality of the PC basis :
βk =
hy(ξ), Ψk (ξ)i
2
,
Ψk
hy(ξ), Ψk i =
Z
y(ξ)Ψk (ξ)pdf (ξ)dξ.
Ω
Numerical integration :
Z
Ω
y(ξ)Ψk (ξ)pdf (ξ)dξ ≈
N
X
i=1
y(ξ (i) )Ψk (ξ(i) )w (i) = β̂k Ψ2k ,
with ξ (i) and w (i) are integration quadrature points / weights.
⊕ Independent computation of the PC coefficients.
Curse of dimension (cubature formula, adpative construction,
Monte-Carlo, . . .)
10/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Uncertainty and sensitivity
analysis by PC
11/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Uncertainty analysis
Uncertainty analysis from PC coefficients is immediate :
The expectation and the variance of the process are given by
E{y(x, t)} = β0 (x, t) and
P∞
2
E{(y(x, t) − E{y(x, t)}) } = k =1 βk2 (x, t)||Ψk ||2 .
Higher moments too.
Fractiles and density estimation can be calculated by
Monte-Carlo simulations of the PC surrogate model
y(x, t, ξ) ≈
P
X
βk (x, t)Ψk (ξ)
k =0
(only polynomials to be computed : not the full model).
12/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Global Sensitivity Analysis
The computation of sensitivity indices from PC coefficients is also
immediate.
Indeed we know exactly the Sobol decomposition of the PCs.
So thanks to orthogonality of the basis and linearity of the PC
expansion one can immediately deduce the Sobol decomposition of
the PC expansion.
13/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Sobol decomposition of the PC surrogate model
For each integrable function f , there is a unique decomposition :
X
f (ξ) =
fu (ξu ), (Sobol1993)
u⊆{1,2,...,d}
with f∅ = f0 .
The Sobol decomposition of a troncated PC expansion ŷ is,
ŷ(ξ) =
X
u⊆{1,2,...,d}
ŷu (ξu ) =
P
X
β̂k Ψk (ξ)
k =0
The terms of the decomposition are
X
β̂k Ψk (ξ)
ŷu (ξu ) =
k ∈Ku
with K = {0, 1, ..., P}, Ku := {k ∈ K |Ψk (ξ) = Ψk (ξ = ξ u )}
and ŷ∅ = β̂0 Ψ0
Thierry Crestaux
SAMO, JUNE 2007
14/23
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Sensitivity indices
Sensitivity indices are calculated with the formula
Su =
Where σŷ2 is
σŷ2 =
σu2
σŷ2
X
u⊆{1,2,...,d}\∅
σu2
and σu2 are explicits for PC expansions
Z
X
β̂k2 ||Ψk ||2
σu2 = ŷu2 (ξ)p(ξ)dξ =
k ∈Ku
15/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Example : Homma-Saltelli
f (ξ) = sin(ξ1 ) + 7sin2 (ξ2 ) + 0.1ξ34 sin(ξ1 ) .
1
L-1 error on sensitivity indices
βk computed by NISP using
Smolyak cubature.
cub S1
cub S2
cub S5
MC S1
MC S2
MC S3
MC S4
MC S5
MC S6
MC S7
0.1
0.01
The figure shows the expectation
of the error on the computation by
Monte-Carlo over 100
simulations.
0.001
1e-04
1e-05
1
10
100
Sample set dimension
1000
10000
F IG .: L-1 error sensitivity indices
computed by PC coefficients and
Monte-Carlo simulation vs. the
sample set dimension
16/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Uncertainty analysis
Sensitivity Analysis
Example : Saltelli-Sobol, non smooth function
g(ξ) =
10
Qp
i=1 (|4ξi
L-1 error on sensitivity indices
1
0.1
0.01
0.001
1e-04
1
10
100
1000
Sample set dimension
− 2| + ai )/(1 + ai ), ai = (i − 1)/2, p = 5 .
βk computed by NISP using
Smolyak cubature.
cub S1
cub S2
cub S3
cub S4
cub S5
MC S1
MC S2
MC S3
MC S4
MC S5
MC S6
MC S7
MC S8
MC S9
MC S10
MC S11
MC S12
MC S13
MC S14
MC S15
10000
The figure shows the expectation
of the error on the computation by
Monte-Carlo over 100
simulations.
100000
F IG .: L-1 error sensitivity indices
computed by PC coefficients and
Monte-Carlo vs. the sample set
dimension
17/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Application :
Advection-dispersion in a porous
media
18/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Equation of advection-dispersion
∂C
∂
(1 + R)θ
(z, t) = −
∂t
∂z
∂C
qC(z, t) − θ(D0 + λ|q|)
(z, t) ,
∂z
(+ Initial and boundary conditions).
Deterministic input
R ≥ 0 decay rate,
q Darcy velocity,
θ ∈]0, 1] porosity,
D0 mol. diffusivity.
Input uncertainties
λ hydrodynamic dispersion coefficient :
λ = aθ b ,
where a and b random
log(a) ∼ U([10−4 , 10−2 ]),
b ∼ U([−3.5, −1]).
19/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Application : Advection-dispersion
t = 5h.
t = 8h.
1000
100
10
10
1
0.1
t = 10h.
100
Galerkin
NISP
1
0
0.2
0.4
0.6
0.8
0.01
1
t = 12h.
100
0
0.2
0.4
0.6
0.8
0.01
1
t = 13h.
C
1
0.1
0.1
0.01
Galerkin
NISP
10
pdf(C)
Galerkin
NISP
100
pdf(C)
pdf(C)
1000
0
100
Galerkin
NISP
0.2
0.4
0.6
0.8
1
t = 14h
C
C
10
Galerkin
NISP
Galerkin
NISP
10
1
pdf(C)
1
pdf(C)
pdf(C)
10
1
0.1
0.1
0.01
0
0.2
0.4
0.6
0.8
t = 15h.
0.1
1
100
0
0.2
0.4
0.6
0.8
t = 18h.
C
0.01
1
0
C
100
Galerkin
NISP
0.2
0.4
0.6
20h.
0.8
1
C
1000
Galerkin
NISP
Galerkin
NISP
100
10
10
1
pdf(C)
pdf(C)
pdf(C)
10
1
1
0.1
0.01
0.1
0
0.2
0.4
0.6
0.8
1
0.1
0
0.2
C
0.4
0.6
0.8
1
0.001
C
0
0.2
0.4
0.6
0.8
1
C
F IG .: Comparison between pdf of the concentration at x = 0.5 for different
times obtained by Galerkin and NISP (no = 6).
Thierry Crestaux
SAMO, JUNE 2007
20/23
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Application : Advection-dispersion
0.8
S_a
S_b
S_ab
0.7
indices de sensibilitØ
0.6
0.5
0.4
0.3
0.2
0.1
0
0
100
200
300
pas de temps
400
500
600
F IG .: Sensitivity indices computed thanks to the PC coefficients computed vs.
time
21/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Conclusion
Summary
Alternative techniques (intrusive / non-intrusive) available for
practicle determination of PC coefficients ;
PC expansion contains a great deal of information in a
convenient compact format ;
Global sensitivity analysis proceeds immediately from PC
expansion ;
Limited to low-moderate dimensionality of the input uncertainty ;
Issues in application to non-smooth processes (remedy : use
non-smooth basis).
22/23
Thierry Crestaux
SAMO, JUNE 2007
Polynomial Chaos expansion
Uncertainty and sensitivity analysis by PC
Application : Advection-dispersion
Conclusion
Perspectives
Improvement of non-intrusive methods (development of efficient
adaptive quadrature techniques, automatic enrichment of sample
sets using active learning techniques) ;
Reduced basis approximation ;
Application to industrial problems ;
Application to identification and optimization problems.
23/23
Thierry Crestaux
SAMO, JUNE 2007