Intro Bayesian Approach Validation Exponential Family Conc
Cross-validation of Controlled Dynamic
Models: Bayesian Approach
M. Kárný, P. Nedoma, V. Šmídl
Adaptive System Department,
Institute of Information Theory and Automation,
Prague.
16th IFAC Congress, July 6, 2005
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
1 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Outline
1
Introduction
Motivation
Aim of the Work
2
Bayesian Approach
Formalizing the Validation Task
Validation by Data Splitting
Bayesian Learning
3
Validation of Dynamic Models
Validation Algorithm
Validation with Multiple Splits
4
Validation of Models from Exponential Family
Computational Efficient Validation
Validation using Stabilized Forgetting
5
Conclusion
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
2 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Motivation
Aim of the Work
Model Validation
Question arising in design of any model or controller:
Is our model good enough?
Possible approaches:
1
2
3
4
test it on the real problem (expensive),
simulation (requires reliable physical model),
expert knowledge (experience, craftsmanship),
formal criteria (limited range of models).
Each is suitable for different problems.
What is the best validation strategy for modelling of large
high-dimensional data archives by mixtures of Gaussian and ARX
models?
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
3 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Motivation
Aim of the Work
Motivation: Static Mixture
eplacements
PSfrag replacements
−10
−5
−10
0
5 Simulated
−5
distribution
Data
10
−2
0
Posterior distribution
5
0
10
−2
2
4
0
2
6
8
4
6
10
12
8
10
Validation:
In this case (2D) we can use expert knowledge (visual
inspection).
What to do in higher-dimensions?
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
4 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Motivation
Aim of the Work
Motivation: Dynamic Mixture (2 ARX models)
Validation using prediction errors:
0.05
0
−0.05
0
100
50
200
150
80
0.2
60
0.1
40
0
20
−0.1
0
−0.05
−0.2
0
0.05
0
250
5
10
300
15
1) expert knowledge, 2) formal criteria, 3) What to do in
higher-dimensions?
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
5 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Motivation
Aim of the Work
Aim of the Work
We seek a formal validation procedure that:
1
2
3
4
fits well into the Bayesian framework,
general enough to cover more complex models,
(Including dynamic models with non-stationary parameters)
use as many standard results as possible,
(we expect that learning procedure is available, at least
approximate)
computationally feasible.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
6 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Formalizing the Validation Task
Learning
System 2
System 1
M∗
bo
bo
M1
M2
finds model, boM ∈ M∗ , which
is the best representation of the
data
(including model structure)
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
7 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Formalizing the Validation Task
Learning
Validation
System 2
System 1
System 2
System 1
M∗V
M∗
M∗
bo
bo
M1
bo
M2
bo
M1
finds model, boM ∈ M∗ , which
is the best representation of the
data
(including model structure)
M. Kárný, P. Nedoma, V. Šmídl
M2
tests assumptions of M∗ in
richer class M∗V , using boM,
)
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
7 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Formalizing the Validation Task
Learning
Validation
System 2
System 1
System 2
System 1
M∗V
M∗
M∗
bo
bo
M1
bo
M2
bo
M1
finds model, boM ∈ M∗ , which
is the best representation of the
data
(including model structure)
M2
tests assumptions of M∗ in
richer class M∗V , using boM,
)
Validation is a ‘poor man’s’ attempt to relax assumptions of M∗
enforced by computational restrictions.
The task is to choose M∗V .
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
7 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Validation by Data Splitting
All available data are split into learning and validation parts.
Learning data
1
Validation data
τ
t̄
learning data bld = d (1 . . . τ ), are used
to identify the model blM,
bv
validation data d = d τ + 1 . . . , t , are used to test predictive
abilities of blM.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
8 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Validation by Data Splitting
All available data are split into learning and validation parts.
Learning data
Validation data
1
t̄
τ
learning data bld = d (1 . . . τ ), are used
to identify the model blM,
bv
validation data d = d τ + 1 . . . , t , are used to test predictive
abilities of blM.
∗
M is a class of single models,
M∗V is a class of models M∗ switching parameters at time τ .
System 1
System 2
Switching M∗
M∗
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
8 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Validation by Data Splitting
All available data are split into learning and validation parts.
Learning data
1
Validation data
τ
t̄
learning data bld = d (1 . . . τ ), are used
to identify the model blM,
bv
validation data d = d τ + 1 . . . , t , are used to test predictive
abilities of blM.
∗
M is a class of single models,
M∗V is a class of models M∗ switching parameters at time τ .
We solve a decision making problem with two hypotheses:
H0 : all data are represented by a single model: boM,
H1 : data sets bld and bvd are generated by different
models, blM and bvM, respectively.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
8 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Formalizing the Validation Task Splitting Bayesian Learning
Bayesian Learning for Dynamic Controlled Models
Probabilistic model:
t
Y
f d 1 . . . t , x 1 . . . t |x0 =
f (yt |ψt , xt ) f (xt |ψt , xt−1 ) f ut |d 1 . . . t ,
t=p
where: data dt = [ut , yt ], outputyt , input ut , parameters (state) xt ,
regressor ψt = ψ d 1 . . . t − 1 , regression length p .
State-estimation: f xt |d 1 . . . t , prior f (x0 ),
Output-prediction:
f yt |ut , d 1 . . . t − 1 ,
Model likelihood:
Qt
L d 1 . . . t , M = t=1 f yt |ut , d 1 . . . t − 1 ,
Model selection:
bo
M = arg minM∈M∗ L d 1 . . . t , M .
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
9 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation Algorithm
All Data
1
M. Kárný, P. Nedoma, V. Šmídl
t̄
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
10 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation Algorithm
f (x0 )
1
1
τ
t̄
Choose model class M∗ , prior f (x0 ), and splitting moment τ .
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
10 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation Algorithm
f (x0 )
bl
1
M
τ
t̄
f (xτ )
1
2
Choose model class M∗ , prior f (x0 ), and splitting moment τ .
Use bld to learn model blM. Choose f (xτ ) (possibly using blM).
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
10 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation Algorithm
f (x0 )
3
bo
M
M
t̄
τ
f (xτ )
2
bl
1
1
bv
M
Choose model class M∗ , prior f (x0 ), and splitting moment τ .
Use bld to learn model blM. Choose f (xτ ) (possibly using blM).
Continue with learning on bvd:
1
2
learn
learn
M. Kárný, P. Nedoma, V. Šmídl
bo
M, starting from blM,
M, starting from f (xτ ).
bv
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
10 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation Algorithm
f (x0 )
3
bo
M
t̄
τ
bv
M
Choose model class M∗ , prior f (x0 ), and splitting moment τ .
Use bld to learn model blM. Choose f (xτ ) (possibly using blM).
Continue with learning on bvd:
1
2
4
M
f (xτ )
2
bl
1
1
learn
learn
bo
M, starting from blM,
M, starting from f (xτ ).
bv
Evaluate likelihoods L (·) of all models and probability
−1
L( bld, blM)L( bvd, bvM)
,
f (H0 |d) = 1 +
L(d, boM)
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
10 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation Algorithm
f (x0 )
3
2
5
bo
M
t̄
τ
bv
M
Choose model class M∗ , prior f (x0 ), and splitting moment τ .
Use bld to learn model blM. Choose f (xτ ) (possibly using blM).
Continue with learning on bvd:
1
4
M
f (xτ )
2
bl
1
1
learn
learn
bo
M, starting from blM,
M, starting from f (xτ ).
bv
Evaluate likelihoods L (·) of all models and probability
−1
L( bld, blM)L( bvd, bvM)
,
f (H0 |d) = 1 +
L(d, boM)
Model class M∗ is valid if f (H0 |d) is close to 1.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
10 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Experiment: ARX model
PSfrag replacements
process:
ARX(2)
model:
ARX(2)
data and pH0
1
pH0
0.5
data
pH0
0
0
50
100
150
200
250
300
200
250
300
time
process:
ARX(4)
model:
ARX(2)
data and pH0
1
0.5
0
data
pH0
−0.5
−1
0
50
100
150
time
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
11 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation with Multiple Splitting Moments
We replace the known splitting moment τ by a grid τ ∗ = [τ1 , . . . , τn ].
Learning
Validation
···
1
τ1
τ2
...
τn
t̄
The number of hypothesis in the decision making problem is much
higher.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
12 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Validation Algorithm Multiple Splits
Validation with Multiple Splitting Moments
We replace the known splitting moment τ by a grid τ ∗ = [τ1 , . . . , τn ].
Learning
Validation
···
1
τ1
τ2
...
τn
t̄
The number of hypothesis in the decision making problem is much
higher.
Can be set up as decision making problem, using loss-functions.
Results:
Maximum-likelihood: choose Hı̂ with
ı̂ = arg max max∗ f (Hi |d) .
i=0,1
τ ∈τ
Marginal-likelihood: treating τ as random variable:
X
L d 1...t ,H =
L d 1 . . . t , H|τ f (τ ) .
τ
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
12 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Computational Efficient Validation Validation using Stabilized Forgetting
Validation in Exponential Family
Estimation of models from exponential family is computationally
attractive, since there exist sufficient statistics (data compression).
V1
V2
Vn
···
1
τ1
τ2
...
τn
t̄
Computational advantages:
1
cheap evaluation
of marginal
likelihoods,
L bld, M = L blV , M .
2
sufficient statistics are additive:
Learning statistics: blV (τ = 2) = V0 + V1 + V2 .
Validation statistics: bvV (τ = 2) = blV0 + V3 + V4 + . . . + Vn+1 .
Overall statistics: boV = V0 + V1 + . . . + Vn+1 .
Validation is only minor computational overhead over learning.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
13 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Computational Efficient Validation Validation using Stabilized Forgetting
Validation using Stabilized Forgetting
In exponential family, stabilized forgetting is defined via hypothesis:
H0 : model parameters are determined by (alternative)
statistics V ,
H1 : model parameters are determined by exponential
window on recent data d (t − k , . . . , t), Vdata .
By setting f (H0 ) = φ, the posterior distribution of parameters is given
by
bo
V = (1 − φ) Vdata + φV .
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
14 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Computational Efficient Validation Validation using Stabilized Forgetting
Validation using Stabilized Forgetting
In exponential family, stabilized forgetting is defined via hypothesis:
H0 : model parameters are determined by (alternative)
statistics V ,
H1 : model parameters are determined by exponential
window on recent data d (t − k , . . . , t), Vdata .
By setting f (H0 ) = φ, the posterior distribution of parameters is given
by
bo
V = (1 − φ) Vdata + φV .
Validation is inverse problem to forgetting:
(alternative) statistics V are chosen as those of the validated
model boV .
parallel run of two learning algorithms with stabilized forgetting:
φ → 1, appropriate for valid model, boV ,
φ → 0, appropriate for invalid model, boV .
test model likelihood of each variant
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
14 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Computational Efficient Validation Validation using Stabilized Forgetting
Application to Mixture Models
Learning:
projection into exponential family (using conditional
independence assumption).
approximate on-line inference of component parameters
(on-line EM, QB, VB, PB),
sensitive to the choice of prior.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
15 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Computational Efficient Validation Validation using Stabilized Forgetting
Application to Mixture Models
Learning:
projection into exponential family (using conditional
independence assumption).
approximate on-line inference of component parameters
(on-line EM, QB, VB, PB),
sensitive to the choice of prior.
Validation:
structure
f (x0 )
f (xτ )
cost
M. Kárný, P. Nedoma, V. Šmídl
‘correct’
estimated
non-informative
non-informative
high
Exponential
fixed
informative
using blM
low
Model Validation: Bayesian Approach
forgetting
fixed
non-informative
non-informative
medium
16th IFAC Congress, July 6, 2005
15 / 16
Intro Bayesian Approach Validation Exponential Family Conc
Conclusion
Model validation was formalized as a special case of model
selection and hypothesis testing,
We have presented a validation method which tests the
assumption of time-invariance of the model,
Advantages:
applicable for all models for which (i) learning procedure, and (ii)
evaluation of data likelihood are available,
computationally efficient algorithm for exponential family,
Disadvantage:
fails to capture some cases of mis-modelling.
M. Kárný, P. Nedoma, V. Šmídl
Model Validation: Bayesian Approach
16th IFAC Congress, July 6, 2005
16 / 16