Lecture 9: Likelihood principle - preliminaries
Lecture 9
Strong Likelihood Principle
The likelihood function contains all the evidence about θ !
• Sufficiency principle
• Conditionality principle
Lecture 9
Sufficiency
Section 3.1 in Pawitan (2001)
An estimate T (x) is sufficient for θ if it summarizes all relevant
information contained in the data about θ .
Example:
1
2
n ∑ xi is sufficient for µ for the data x1 , x2 , ..., xn sampled from iid N(θ , σ ).
Lecture 9
Sufficiency
Section 3.1 in Pawitan (2001)
An estimate T (x) is sufficient for θ if it summarizes all relevant
information contained in the data about θ .
Example:
1
2
n ∑ xi is sufficient for µ for the data x1 , x2 , ..., xn sampled from iid N(θ , σ ).
Lecture 9
Sufficiency Principle
All suffcient statistics based on the data x should lead to the same
conclusion for a given model.
(Page 194 in Pawitan 2001)
Lecture 9
The Likelihood Function is Minimal Sufficient
A suffcient statistic T (x) is minimal sufficient if it is a function of any
other sufficient statistic. (Page 55 in Pawitan 2001)
Theorem 3.2 (page 56 in Pawitan 2001)
If T is sufficient for θ in an experiment E then the likelihood of θ based on the
whole data x is the same as that based on T alone.
Example 3.3:
Suppose x1 , x2 , ..., xn are an iid sample from iid N(θ , 1). Then the likelihood is
n
1
1
L(θ ; x1 , x2 , ..., xn ) = √
exp − ∑(xi − θ )2
2
2π
and the log-likelihood can be written as
logL(θ ; x1 , x2 , ..., xn ) = −
n
n
1
(xi − x̄)2 − (x̄ − θ )2 = constant − − (x̄ − θ )2 .
∑
2
2
2
Hence, the likelihood based on the whole data set is the same as the likelihood as
that based on x̄ alone.
Theorem 3.2 implies that The likelihood function is minimal sufficient.
Lecture 9
The Likelihood Function is Minimal Sufficient
A suffcient statistic T (x) is minimal sufficient if it is a function of any
other sufficient statistic. (Page 55 in Pawitan 2001)
Theorem 3.2 (page 56 in Pawitan 2001)
If T is sufficient for θ in an experiment E then the likelihood of θ based on the
whole data x is the same as that based on T alone.
Example 3.3:
Suppose x1 , x2 , ..., xn are an iid sample from iid N(θ , 1). Then the likelihood is
n
1
1
L(θ ; x1 , x2 , ..., xn ) = √
exp − ∑(xi − θ )2
2
2π
and the log-likelihood can be written as
logL(θ ; x1 , x2 , ..., xn ) = −
n
n
1
(xi − x̄)2 − (x̄ − θ )2 = constant − − (x̄ − θ )2 .
∑
2
2
2
Hence, the likelihood based on the whole data set is the same as the likelihood as
that based on x̄ alone.
Theorem 3.2 implies that The likelihood function is minimal sufficient.
Lecture 9
The Likelihood Function is Minimal Sufficient
A suffcient statistic T (x) is minimal sufficient if it is a function of any
other sufficient statistic. (Page 55 in Pawitan 2001)
Theorem 3.2 (page 56 in Pawitan 2001)
If T is sufficient for θ in an experiment E then the likelihood of θ based on the
whole data x is the same as that based on T alone.
Example 3.3:
Suppose x1 , x2 , ..., xn are an iid sample from iid N(θ , 1). Then the likelihood is
n
1
1
L(θ ; x1 , x2 , ..., xn ) = √
exp − ∑(xi − θ )2
2
2π
and the log-likelihood can be written as
logL(θ ; x1 , x2 , ..., xn ) = −
n
n
1
(xi − x̄)2 − (x̄ − θ )2 = constant − − (x̄ − θ )2 .
∑
2
2
2
Hence, the likelihood based on the whole data set is the same as the likelihood as
that based on x̄ alone.
Theorem 3.2 implies that The likelihood function is minimal sufficient.
Lecture 9
The Likelihood Function is Minimal Sufficient
A suffcient statistic T (x) is minimal sufficient if it is a function of any
other sufficient statistic. (Page 55 in Pawitan 2001)
Theorem 3.2 (page 56 in Pawitan 2001)
If T is sufficient for θ in an experiment E then the likelihood of θ based on the
whole data x is the same as that based on T alone.
Example 3.3:
Suppose x1 , x2 , ..., xn are an iid sample from iid N(θ , 1). Then the likelihood is
n
1
1
L(θ ; x1 , x2 , ..., xn ) = √
exp − ∑(xi − θ )2
2
2π
and the log-likelihood can be written as
logL(θ ; x1 , x2 , ..., xn ) = −
n
n
1
(xi − x̄)2 − (x̄ − θ )2 = constant − − (x̄ − θ )2 .
∑
2
2
2
Hence, the likelihood based on the whole data set is the same as the likelihood as
that based on x̄ alone.
Theorem 3.2 implies that The likelihood function is minimal sufficient.
Lecture 9
The weak likelihood principle
Any set of observations from a given model pθ (x) with the same likelihood should
lead to the same conclusion.
(Page 194 in Pawitan 2001)
Lecture 9