ISIPTA’07
July 19th, 2007
Data-Based Decisions under
Imprecise Probability and Least
Favorable Models
Robert Hable ∗
Department of Statistics
LMU Munich
∗
supported by Cusanuswerk
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Robert Hable
Department of Statistics, LMU Munich (Germany)
I
Education: Mathematics (in Bayreuth, Gemany)
I
Diploma Thesis in Robust Asymptotic Statistics
(Helmut Rieder)
I
Now: Ph.D. Student under the Guidance of Thomas Augustin
→ Topic: “Data-Based Decisions under Complex Uncertainty”
Research Interests:
I
I
I
I
Decision Theory under Imprecise Probabilities
Mathematical Foundations of Imprecise Probabilities
Robust Statistics
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Usual Decision Theory
I
I
I
States of nature: Θ = {θ1 , . . . , θn }
Decisions: t ∈ D
Bounded loss functions: Wθ : D → R , t 7→ Wθ (t)
t1
..
.
tk
..
.
tm
Robert Hable
θ1
Wθ1 (t1 )
···
..
..
.
θi
···
···
.
Wθi (tk )
..
Wθ1 (tm )
θn
Wθn (t1 )
···
.
Wθn (tm )
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Usual Decision Theory
I
I
I
I
I
States of nature: Θ = {θ1 , . . . , θn }
Decisions: t ∈ D
Bounded loss functions: Wθ : D → R , t 7→ Wθ (t)
Prior distribution over Θ : π = (πθ1 , . . . , πθn )
P
Expected loss for decision t ∈ D : θ∈Θ πθ Wθ (t)
Often: Decision making on base of observations y ∈ Y
I
I
I
Decision functions: δ : Y → D , y 7→ δ(y )
Distribution of the observation y : qθ where θ ∈ Θ
Expected loss for decision function δ : Y → D is
X Z
πθ Wθ (δ(y )) qθ (dy )
θ∈Θ
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Decision Theory under Imprecise Probability
I
Instead of a precise prior distribution π:
Imprecise prior distribution (coherent upper prevision):
Π[f ] = sup π[f ] ,
f :Θ→R
π∈P
P: a set of precise prior distributions (credal set)
I
Instead of a precise distribution qθ of the observation y:
Imprecise distribution of the observation y (coherent upper
prevision):
Q θ [g ] = sup qθ [g ]
qθ ∈Mθ
∀θ ∈ Θ,
g :Y→R
Mθ : a set of precise distributions of the observation y
(credal set)
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Decision Theory under Imprecise Probability
I
Imprecise prior distribution (coherent upper prevision):
Π[f ] = sup π[f ]
π∈P
I
Imprecise distribution of the observation y (coherent upper
prevision):
Q θ [g ] = sup qθ [g ]
∀θ ∈ Θ
qθ ∈Mθ
Upper expected loss for decision function δ : Y → D is
Z
X
sup
πθ sup
Wθ (δ(y )) qθ (dy )
π∈P θ∈Θ
Robert Hable
qθ ∈Mθ
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Task
Find a decision function δ̃ which minimizes the upper expected loss
Z
X
sup
πθ sup
Wθ (δ(y )) qθ (dy ) = min!
π∈P θ∈Θ
qθ ∈Mθ
δ
Optimality criterion:
Γ-minimax criterion: worst-case consideration
Problem:
often, direct solution computationally intractable
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Common Idea
Find another optimization problem which has the following
properties:
I
Solving this new optimization problem leads to a solution of
the original problem.
I
The new optimization problem should be computationally
tractable!
−→ Least Favorable Models
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Least favorable model
I
Mθ : credal set for the distribution of the observation y
I
P : credal set for the prior distribution
I
Find some q̃θ ∈ Mθ for every θ ∈ Θ so that
Z
X
inf
πθ sup
Wθ (δ(y )) qθ (dy ) =
δ
qθ ∈Mθ
θ∈Θ
X
= inf
δ
q̃θ
θ∈Θ
Z
πθ
Wθ (δ(y )) q̃θ (dy )
∀π ∈ P
θ∈Θ
∈ Mθ
θ∈Θ
is called least favorable model.
(−→ Huber-Strassen (1973))
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Then, we have:
The new optimization problem
Z
X
sup
πθ sup
Wθ (δ(y )) q̃θ (dy ) = min!
π∈P θ∈Θ
qθ ∈Mθ
δ
is computationally easier than the original optimization problem
Z
X
sup
πθ sup
Wθ (δ(y )) qθ (dy ) = min!
π∈P θ∈Θ
Robert Hable
qθ ∈Mθ
δ
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
. . . and we have:
There is a solution δ̃ of the new optimization problem
Z
X
sup
πθ sup
Wθ (δ(y )) q̃θ (dy ) = min!
π∈P θ∈Θ
qθ ∈Mθ
δ
which also solves the original optimisation problem is
Z
X
sup
πθ sup
Wθ (δ(y )) qθ (dy ) = min!
π∈P θ∈Θ
Robert Hable
qθ ∈Mθ
δ
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
However
A least favorable model q̃θ
exist!
θ∈Θ
∈ Mθ
θ∈Θ
does not always
That is: The presented procedure does not always work!
Question: When does it work?
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Main Result
The Main Theorem provides:
A necessary and sufficient
condition
for the existence of a least
favorable model q̃θ θ∈Θ ∈ Mθ θ∈Θ
Remarks:
I
exact condition is rather involved;
uses some of Le Cam’s concepts such as
I
I
I
equivalence of models
conical measures (or standard measures)
generalizes Buja (1984) and Huber-Strassen (1973)
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Comparison with Buja 1984 – Some Technicalities
Buja 1984
I
Credal sets Mθ only contain σ-additive probability measures
I
Condition: Compactness of credal sets Mθ
This is restrictive in Buja’s setup! (cf. Hable (2007B, E-print ))
Now
I
Credal sets Mθ may contain finitely-additive probability
measures (which are not σ-additive).
I
Compactness of credal sets Mθ comes for free in Walley’s
setup.
A first conclusion:
I
σ-additivity is not necessary here.
I
Getting around σ-additivity is possible by Le Cam’s setup
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
Le Cam’s setup
I
I
I
Le Cam: strictly functional analytic approach to probability
theory (cf. e.g. Hable (2007C, E-print ))
Rather involved and abstract (uses advanced functional
analytic methods)
“Traditional concepts” (σ-additivity, Markov-kernels,. . . ):
appropriate for small models (dominated by a σ-finite
measure)
Le Cam’s concepts: also appropriate for large models
A second conclusion:
Imprecise probabilities lead to large models
−→ Le Cam’s concepts are appropriate for the theory of
imprecise probabilities.
−→ Maybe, they could/should be used further on.
Robert Hable
LMU Munich
Data-Based Decisions under Imprecise Probability and Least Favorable Models
References
I
I
I
Buja, A. (1984): Simultaneously Least Favorable Experiments,
Part I: Upper Standard Functionals and Sufficiency. Zeitschrift
für Wahrscheinlichkeitstheorie und verwandte Gebiete 65,
367–384.
Hable, R. (2007A): Data-Based Decisions under Imprecise
Probability and Least Favorable Models. In: De Cooman, G.,
Vejnarová, J., Zaffalon, M. (eds.). Proceedings of the Fifth
International Symposium on Imprecise Probability: Theories
and Applications, 203–212.
Hable, R. (2007B): A Note on an Error in Buja (1984).
E-print.
www.statistik.lmu.de/˜hable/publications.html
I
Hable, R. (2007C): Decision Theory: Some of Le Cam’s
Concepts. E-print.
www.statistik.lmu.de/˜hable/publications.html
Robert Hable
LMU Munich