Motivation/Background
Parameter estimation
Prediction
Summary
The consequences of misspecifying the
random effects distribution when fitting
generalized linear mixed models
John M. Neuhaus
Charles E. McCulloch
Division of Biostatistics
University of California, San Francisco
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Outline
1
Motivation/Background
2
Parameter estimation
3
Prediction
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Antibotic Rx Study in acute care
(Gonzales et al, Academic Emergency Medicine 2006)
Study objectives: 1) assess intervention to reduce
inappropriate antibiotic prescription rates - Rx for
antibiotic-nonresponsive conditions (e.g. acute bronchitis);
2) identify poorly performing providers
Design: multiple responses from 720 providers (clusters)
Large between-provider variability in RX rates & cluster
sizes - borrow strength between providers
Popular approach: Use estimated regression coefficients &
predicted random effects from GLMMs
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Generalized Linear Mixed Models
Yij | bi , Xij ∼ GLM
E(Yij |Xij , bi ) = h−1 (β0 + bi + β1 Xij )
h(·) link function, e.g. identity, logit, probit
i=1,. . . , m : subjects;
j=1,. . . , ni : repeats per cluster
When b ∼ GT , Yi1 , . . . , Yni cond’ly indep given bi & bi indep of X
Likelihood: L(β, GT ) =
ni
m Z Y
Y
i=1
John M. Neuhaus, Charles E. McCulloch
f (Yij | b, Xij ) dGT (b)
j=1
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Misspecified random effects distributions
GT typically unknown & we might assume b ∼ GF 6= GT . Often
GF = N(0, σb2 ) (e.g. Stata default)
Two general forms of misspecified GT :
Incorrect distributional shape
Incorrectly assuming b indep of X
1
E(b | X ) = µM (X ), Neuhaus & McCulloch (2006)
2
var (b | X ) = µV (X ), Heagerty & Kurland (2001)
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Worry about correctly specifying the shape of GT ?
Some say yes - Motivation for more flexible GF , e.g.
mixture of normals (Chen et al 2002), nonparametric GF
(Lesperance & Kalbfleisch 1992) & specification tests
(Tchetgen & Coull 2006)
Others show little bias in “ slopes", β̂1 (Neuhaus et al 1992)
Little work on random effects prediction under
misspecification
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Bias with misspecified shape of G?
Linear mixed effects model: Yij | bi = β0 + σb bi + β1 Xij + eij
b ⊥ e, b ∼ GT , E(b) = 0, var (b) = 1, var (e) = σe2
cov (Yi1 , . . . , Yini ) = V = σe2 I + σb2 J, indep of GT
E{β̂GLS } = E{(X T V −1 X )−1 X T V −1 Y } = β1 , indep of GF
Unbiased estimators of slopes, β1 , with misspecified shape
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Assessing consequences of random effects
misspecification
Follow theory on inference with misspecified models
(e.g. White 1994) - let ξ = (β, θ)
True : fT (Yi | Xi ; ξ) =
Z Y
ni
f (Yij | b, Xij ; β) dGT (b; θ)
j=1
Fitted : fF (Yi | Xi ; ξ ∗ ) =
Z Y
ni
f (Yij | b, Xij ; β ∗ ) dGF (b; θ∗ )
j=1
“ MLE" ξˆ∗ → ξ ∗ minimizes EX EY |X log {fT (y |X ; ξ)/fF (y |X ; ξ ∗ )}
Kullback-Leibler Divergence
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Misspecified models
Minimizing K-L Divergence wrt ξ ∗ = (ξ1∗ , . . . , ξq∗ ) yields
Z Y
ni
∂
{
f (Yij | b, Xij ; β ∗ ) dGF (b; θ∗ )}dy = 0
λ(y |X , ξ )
∂ξk ∗
y
Z
EX
∗
j=1
where λ(y |X , ξ ∗ ) = fT (Y = y |X , ξ)/fF (Y = y |X , ξ ∗ )
Note: ξ ∗ that yield λ(y |X , ξ ∗ ) = 1 ∀X solve system
Can solve for ξ ∗ analytically in some simple cases, i.e. match
fitted & true densities at all points X
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Matched pairs, solutions under misspecification
For some link functions can find analytic solution with
β1∗ = β1 , but β0∗ 6= β0 & σ ∗ 6= σ
Special cases: β̂1∗ consistent, but β̂0∗ & σ̂ ∗ inconsistent
when GF 6= GT
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Logistic link
Binary matched pairs - when GF generates a wide range of
pr (y1 , y2 ) (e.g. GF = Normal) β̂1∗ is consistent (&
β̂1∗ ≡ β̂CML ) (Neuhaus et al 1994)
GF unspecified, β̂1∗ ≡ β̂CML (Lindsay et al 1991)
General X - when true β1 = 0, β1∗ = 0 solves
Kullback-Leibler minimizing equations ⇒ β̂1∗ consistent
Typically cannot solve Kullback-Leibler system analytically
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Numerical solution of Kullback-Leibler equations
Numerically find ξ ∗ that minimizes
EX EY |X log {fT (y |ξ, X )/fF (y |ξ ∗ , X )} using routines in R
Numerically evaluate integrals
Numerically minimize Kullback-Leibler divergence
Allows evaluation of bias over wide range of parameter
values, random effect & covariate distributions
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Numerical assessment of bias
True: logit {pr (Yij = 1|Xij , bi )} = β0 + σbi + β1 Xij
True GT : b ∼ exp(1), shifted to E(b) = 0
Fitted GF : b ∼ N(0, 1)
X varied within clusters X = (0, .25, .5, .75, 1), ni = 5
Range of parameter values:
β0 : (-3,-1) by 0.1
β1 : (0.5,2) by 0.1
σ: (0.2,2.5) by 0.1
Numerically solved for β0∗ , β1∗ , σ ∗ that minimize Kullback-Leibler
divergence
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Bias in beta1, True = exponential, Fitted = Normal
beta0: −3
beta0: −2
beta0: −1
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
10
0
sigma: 0.2
5
−5
−10
10
0
beta0
sigma: 1
% Bias
5
−2
−3
−5
−10
10
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
0.6 0.8 1 1.2 1.4 1.6 1.8
0.6 0.8 1 1.2 1.4 1.6 1.8
0.6 0.8 1 1.2 1.4 1.6 1.8
−5
−10
True beta1
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
sigma: 2.5
5
0
−1
Motivation/Background
Parameter estimation
Prediction
Summary
Bias in sigma, true = exponential, fitted = Normal
beta0: −3
50
●
●
40
beta0: −1
●
●
●
●
●
● ● ● ● ●
● ●
● ● ●
● ● ●
● ● ● ● ●
● ● ●
10
● ● ● ● ● ●
● ●
● ● ●
● ● ● ● ●
● ● ● ● ● ● ● ●
0
beta1: 0.5
30
20
beta0: −2
● ● ● ● ●
● ●
● ●
● ●
● ●
● ● ●
● ● ●
●
−10
50
40
●
●
% Bias
20
●
● ● ● ● ●
● ●
● ●
● ●
● ● ●
● ● ●
● ● ●
beta0
●
●
●
10
beta1: 1
30
● ● ●
● ●
● ●
● ● ●
● ● ● ●
● ● ● ● ● ● ●
●
0
−10
50
40
●
20
●
● ● ● ●
● ●
● ● ●
● ● ●
● ● ● ●
● ● ● ● ●
beta1: 1.9
30
●
10
●
0
● ● ● ● ● ● ●
● ● ●
● ● ● ●
● ● ● ● ● ● ● ● ●
● ●
●
−10
0.5
1
1.5
2
2.5
0.5
1
1.5
2
2.5
0.5
True sigma
John M. Neuhaus, Charles E. McCulloch
● ●
● ●
● ●
● ● ●
● ● ● ● ● ● ● ● ● ● ● ●
Stata Users 2008
1
1.5
2
2.5
−1
−2
−3
● ● ● ●
● ●
● ●
● ● ●
● ● ● ● ●
● ● ● ● ● ● ● ●
Motivation/Background
Parameter estimation
Prediction
Summary
Bias in beta0, true = exponential, fitted = Normal
sigma: 0.2
50
sigma: 1
sigma: 2.5
●
●
40
●
●
●
beta1: 0.5
● ●
● ●
● ● ● ● ● ● ● ● ● ● ● ●
30
20
● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
10
0
50
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
40
% Bias
30
●
●
●
sigma
beta1: 1
● ●
● ●
● ● ● ● ● ● ● ● ● ● ● ●
20
● ● ● ● ●
●
● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
40
●
●
● ●
● ●
● ● ● ● ● ● ● ● ● ● ● ●
●
●
beta1: 1.9
30
●
20
10
0
1
0.2
10
0
50
2.5
●
● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
−3
−2.5
−2
−1.5
−1 −3
−2.5
−2
−1.5
−1 −3
True beta0
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
−2.5
−2
−1.5
−1
Motivation/Background
Parameter estimation
Prediction
Summary
Logistic link - further results
Find approximate solution of Kullback-Leibler equations
using Taylor series methods (Neuhaus et al 1992)
Approximate solution & simulations indicate E(β̂1 ) ≈ β1
when GT misspecified, but biased β̂0 & σ̂b
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Antibiotic Rx of acute respiratory infections
Study objective: assess intervention to reduce
inappropriate antibiotic prescription rates - Rx for
antibiotic-nonresponsive conditions (e.g. acute bronchitis)
Cluster: visits to a provider for antibiotic-nonresponsive
conditions - number of visits varied from 1 to 71
Design: Baseline Y → intervention → Post-Y
Binary outcome: prescribed antibiotics for nonresponsive
conditions (yes/no) - measured at each visit
Covariates: intervention, time, time*intervention, provider
type, illness duration prior to visit
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Antibiotic Rx study - Fitted models
logit pr (Yij = 1|b, Xij ) = β0 + exp(log σb ) b + β1 Xij
where b ∼ G with E(b) = 0, Var (b) = 1
G1 = N(0,1)
G2 = Exponential (1) standardized to E(b) = 0
G3 = Nonparametric, 4 support points (GLLAMM)
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Antibiotic Rx study - Results
Estimated parameters from mixed-effects logistic models with
different random effects distributions
β̂, (SE)
G
Normal
Exponential
NP (4)
β0
-1.06
(0.53)
-0.92
(0.52)
-1.01
(0.51)
TREAT
0.66
(0.32)
0.72
(0.31)
0.66
(0.32)
John M. Neuhaus, Charles E. McCulloch
TIME
0.56
(0.32)
0.55
(0.31)
0.59
(0.31)
TRT*TIME
-0.52
(0.20)
-0.53
(0.19)
-0.55
(0.19)
Stata Users 2008
log σb
0.09
(0.08)
0.14
(0.09)
0.02
Motivation/Background
Parameter estimation
Prediction
Summary
Summary for fixed effects estimation
1
Misspecifying shape of GT yields accurate “ slopes" β̂1
over wide range of β0 , σb2
2
Misspecifying shape of GT can yield biased estimates of
β0 & σb2
• %bias in β̂0 ↑ with σb2 & can be substantial
• % bias in σ̂ can also be large
3
When interest focuses on slope parameters, misspecifying
shape has little effect on bias
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Prediction of random effects
Useful method: Predict b by b̃ that minimize
Z Z
E[b̃ − b]2 =
(b̃ − b)2 f (b, y ) dy db,
where f (b, y ) is the joint density of b & y1 , . . . , yn
Can show: b̃i = E(bi |yi1 , . . . , yini )
Depends on f (b, y ), hence on G
Misspecifying G may produce inaccurate b̃
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Linear Mixed-Effects Model
Yij = β0 + bi + β1 Xij + ij ,
i = 1, . . . , m; j = 1, . . . , ni
bi ∼ N(0, σb2 ), ij ∼ N(0, σ2 )
Estimated BLUP: Estimated b̃ = b̂i = D̂i ZiT V̂i−1 (yi − Xi β̂)
ˆ (bi ) V̂i = var
ˆ (yi )
D̂i = Cov
Expression depends on joint normality of b & y
Misspecification of distribution of b may produce inaccurate b̃
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Theory for Linear Mixed-Effects Model (simple case)
Yij = µ + bi + ij ,
i = 1, . . . , m; j = 1, . . . , ni
bi ∼ N(0, σb2 ), ij ∼ N(0, σ2 ), ij ⊥ bi , µ, σb2 , σ2 known
Best Linear Unbiased Predictor (BLUP) is b̃i that minimizes
E[(b̃i − bi )2 ]
Simple calculations show that
b̃i = E[bi |y ] =
σb2
σb2
(
Ȳ
−
µ)
=
(bi + ¯i· )
i·
σb2 + σ2 /ni
σb2 + σ2 /ni
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Theory for LME (cont.)
Conditional on bi , the Yij are independent N(µ + bi , σ2 )


σ2
b̃i | bi ∼ N µb̃ = 2 b 2
bi ,

σb + σ /ni
σb2
σb2 + σ2 /ni
!2

σ2 
ni 
Thus, b̃i is conditionally biased
However, since calculations are |bi , result does not depend on
distribution of bi
i.e. conditional bias does not depend on distribution of bi
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Features of b̃i
As ni → ∞,
b̃i =
σb2
σb2
(b
+
¯
)
→
(bi + 0) = bi
i
i·
σb2 + σ2 /ni
σb2
⇒ b̃i converges to true value as ni → ∞
Such asymptotics typically not of interest, rather as m → ∞
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Density of b̃i
What does the density of b̃i look like?
Also, what if b̃i misspecified to be normal?
If ni large, then each b̃i ≈ bi ⇒ density is approximately
correct, irrespective of assumed density
What if ni not large, usual case of interest?
Then density of b̃i is convolution of true density of bi with the
conditional density of b̃i given bi
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Density of b̃i
What does the density of b̃i look like?
Also, what if b̃i misspecified to be normal?
If ni large, then each b̃i ≈ bi ⇒ density is approximately
correct, irrespective of assumed density
What if ni not large, usual case of interest?
Then density of b̃i is convolution of true density of bi with the
conditional density of b̃i given bi
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Density of b̃i
What does the density of b̃i look like?
Also, what if b̃i misspecified to be normal?
If ni large, then each b̃i ≈ bi ⇒ density is approximately
correct, irrespective of assumed density
What if ni not large, usual case of interest?
Then density of b̃i is convolution of true density of bi with the
conditional density of b̃i given bi
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Density of b̃i
What does the density of b̃i look like?
Also, what if b̃i misspecified to be normal?
If ni large, then each b̃i ≈ bi ⇒ density is approximately
correct, irrespective of assumed density
What if ni not large, usual case of interest?
Then density of b̃i is convolution of true density of bi with the
conditional density of b̃i given bi
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Density of b̃i (cont)
Suppose bi ∼ Exponential(1), shifted so that E(bi ) = 0
Density of b̃i (under normal assumption) is
Z ∞
exp{−(b̃ − µb̃ )2 ni /(2σ2 )} exp(−b̃ − 1) d b̃
0
Straightforward to evaluate numerically
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 2
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 4
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 6
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 8
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 10
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 16
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
0.6
0.7
Plot of BLUP Densities for Cluster Size 20
0.3
0.2
0.1
0.0
Density
0.4
0.5
Assumed Normal
Exponential
−2
−1
0
1
2
BLUP
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
BLUP density plot findings
Density of BLUPs inherits much of its shape from assumed
density
Doesn’t reflect shape of true density of the random effects
until cluster sizes get large
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Best Predicted values
Under LME, with eij ∼ N(0, σe2 ), & true density fbi (bi ),
R∞
b̃i = E[bi |Yi ] =
ni
2
−∞ bi exp{− 2σ2 (νi − bi ) }fbi (bi )dbi
R∞
ni
2
−∞ exp{− 2σ2 (νi − bi ) }fbi (bi )dbi
where νi = (Ȳi· − x̄0i· β)
Given νi , compute b̃i for various true fbi (bi ): Normal, T3 ,
Exponential (1), Mixture of Normals
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
−5
Best Prediction
0
5
BP plots
−5
−4
−3
−2
−1
0
1
Within Cluster Deviation
Normal
Mixture
John M. Neuhaus, Charles E. McCulloch
2
T w/ 3 d.f.
Exponential
Stata Users 2008
3
4
5
Motivation/Background
Parameter estimation
Prediction
Summary
Order preservation for LME
Can show
monotone
∂ b̃i
∂νi
> 0 for any fbi (bi ) ⇒ transformation νi → b̃i
Given ni , BP’s under any assumed f ordered based on νi
Let f1 (bi ), f2 (bi ) denote assumed random effects densities
b̃i1 (νi ) & b̃i2 (νi ), predictions from each value νi
Order the pairs [b̃i1 (νi ), b̃i2 (νi )] by νi
Since
∂ b̃i
∂νi
> 0, pairs also ordered by b̃i1 (νi ) & b̃i2 (νi ).
Thus, all pairs concordant & Kendall’s τ between b̃i1 (νi )
and b̃i2 (νi ) is 1
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Mean square error of prediction
Given σb2 , σe2 , we calculated b̃ assuming
b ∼N(0,1)
b ∼ Exponential
b ∼ Mixture of two N(0,1)
Using expressions for b̃ we calculated MSE = E[(b̃ − b)2 ] for
various true fb (b) using numerical integration
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
True Exponential Random Effects
40
.6
MSEP
.4
.2
10
20
30
Sample Size Per Cluster
40
40
MSEP
.4
40
.8
10
20
30
Sample Size Per Cluster
40
0
10
20
30
Sample Size Per Cluster
40
Random Effects SD 2
0
.2
MSEP
.4
.6
MSEP
.4
.2
0
40
.2
10
20
30
Sample Size Per Cluster
Random Effects SD 2
0
0
.2
MSEP
.4
.6
.8
0
.8
10
20
30
Sample Size Per Cluster
Random Effects SD 2
10
20
30
Sample Size Per Cluster
Random Effects SD 1
.6
0
0
0
.2
MSEP
.4
.6
.8
Random Effects SD 1
0
0
.2
MSEP
.4
.6
.8
0
.8
10
20
30
Sample Size Per Cluster
Random Effects SD 1
Random Effects SD 0.5
.6
0
True Mixture Random Effects
0
.2
MSEP
.4
.6
.8
Random Effects SD 0.5
0
0
.2
MSEP
.4
.6
.8
Random Effects SD 0.5
.8
True Gaussian Random Effects
0
10
20
30
Sample Size Per Cluster
40
0
10
20
30
Sample Size Per Cluster
40
Assumed distributions: Solid line/square=Gaussian, dotted line/circle=Exponential, dashed line/X=Mixture
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Simulations
Evaluate performance of BP’s when all parameters estimated
Two true & assumed f (b), i.e. 4 true/assumed settings
b ∼ N(0, 1)
b ∼ Tukey(g = 0.5, h = 0.1)
Linear mixed effects & mixed effects logistic models
Linear predictor: ηij = −2 + bi + xbetween + xwithin
σb2 = 1, σe2 = 1 for LME
ˆ = (1/mK ) PK
Calculated MSE
k =1
Pm
i=1 (b̃ki
− bki )2
K = 1000, m = 100, cluster sizes=2, 4, 6, 10, 20, 40
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Percentiles of N(0,1) & standardized Tukey(0.5,0.1)
Percentile
.1%
1%
2.5%
5%
10%
50%
90%
95%
97.5%
99%
99.9%
N(0,1)
-3.09
-2.33
-1.96
-1.64
-1.28
0
1.28
1.64
1.96
2.33
3.09
Standardized Tukey(0.5,0,1)
-1.89
-1.40
-1.21
-1.06
-0.89
-0.21
1.09
1.73
2.47
3.62
7.68
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Continuous Outcome
Binary Outcome
1
True Gaussian
MSEP
0
0
.2
.1
.4
MSEP
.2
.6
.3
.8
.4
True Gaussian
0
10
20
Cluster size
30
40
0
10
20
Cluster size
30
40
30
40
1
True Tukey
MSEP
0
0
.2
.1
.4
MSEP
.2
.6
.3
.8
.4
True Tukey
0
10
20
Cluster size
30
40
0
10
Assumed distributions: Solid line/square=Gaussian, dotted line/circle=Tukey
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
20
Cluster size
Motivation/Background
Parameter estimation
Prediction
Summary
Antibiotic Rx study - Fitted models
Objective: identify poorly performing providers (i.e. large
predicted random effects)
logit pr (Yij = 1|b, Xij ) = β0 + exp(log σb ) b + β1 Xij
where b ∼ G with E(b) = 0, Var (b) = 1
G1 = N(0,1)
G2 = Exponential (1) standardized to E(b) = 0
Compute b̃ that maximizes
log{f (yi1 , . . . , yini | Xi1 , . . . , Xini , β, bi ) g(bi | θ)}
Fit models using PROC NLMIXED in SAS
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Predicted random effects from normal & exponential
●
●
●
●
2
●
●
●
Spearman corr=0.99
●
●●
●
●
●
● ●
● ●
●
●●
●
●●●
●● ●
● ●
●
●
●
●
● ●
● ●●● ●●
●
●
● ● ●
●●
●
●●
●
● ●●
● ●
●
●
●
●
● ● ●
●
●
●
●
●●●
●
● ●
● ●
● ●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
● ●
●●
●● ●
●
●● ●
●
●
●
●●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
● ●●
● ● ●
● ●●
●●
●
●
●
●
●●
●
●
1
0
●
●
−1
b.exp
●
−1.5
−1.0
−0.5
0.0
0.5
b.norm
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
1.0
1.5
2.0
Motivation/Background
Parameter estimation
Prediction
Summary
Summary of effects on prediction
1
Predicted values of random effects show modest sensitivity
to assumed distributional shape.
2
Distribution shape of BLUPs often not reflective of true
random effects distribution.
3
Ranking of predicted values is little affected.
4
Misspecified shape can produce modest increases in MSE
of prediction.
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008
Motivation/Background
Parameter estimation
Prediction
Summary
Summary
Misspecifying the shape of the random effects produces
little bias in estimated covariate effects
slight deterioration in random effects prediction.
Stata default of b ∼ N(0, σb2 ) yields accurate estimates of
covariate effects & reasonably precise predicted random effects
John M. Neuhaus, Charles E. McCulloch
Stata Users 2008