Part 2: Linear models for continuous data
These methods use the assumption that the responses are multivariate normal, but do not require it.
Notation:
Yi,j = j th measurement on ith subject,
made on occasion ti,j , j = 1, 2, . . . , ni, i = 1, 2, . . . , N .
vector:


Yi,1

 Y
 i,2 
Yi =  .. 
 . 
Yi,ni
As a
1
Covariates:
• Associated with Yi,j : covariates Xi,j,k , k = 1, 2, . . . , p


Xi,j = 


Xi,j,1
Xi,j,2
...
Xi,j,p





• Some covariates (e.g. gender, treatment) do not change with
occasion (do not depend on j).
• Others do (e.g. time since baseline, smoking status).
2
Matrix of covariates (ni × p):
X0i,1

 X0i,2
Xi = 

...

X0i,ni




 , i = 1, 2, . . . , N.


Linear model:
Yi,j = β1Xi,j,1 + β2Xi,j,2 + · · · + βpXi,j,p + ei,j , j = 1, 2 . . . , ni.
or
Yi = Xiβ + ei.
3
Example: TLC
Balanced:
• N = 100
• ni = n = 4, i = 1, 2, . . . , N
• ti,j = tj , i = 1, 2, . . . , N , j = 1, 2, 3, 4
• t1 = 0, t2 = 1, t3 = 4, t4 = 6.
4
Model: linear change over time, common intercept, different
slope in each group:
Yi,j = β1Xi,j,1 + β2Xi,j,2 + β3Xi,j,3 + ei,j
= X0i,j β + ei,j .
• Xi,j,1 = 1 for all i and j, so β1 is the intercept.
• Xi,j,2 = tj
• and

t
Xi,j,3 = j
0
if subject i is in the succimer group,
if subject i is in the placebo group
5
We assume the errors ei,j have mean zero, so
E(Yi,j ) = X0i,j β

β + β t
1
2 j
=
β + (β + β )t
1
2
3 j
if subject i is in the placebo group,
if subject i is in the succimer group,
and β3 is the difference in slopes.
6
Distributional Assumption
Yi has the multivariate normal distribution M V Nni (µi, Σi), with
µi = Xiβ .
The probability density function for M V Nn(µ, Σ) is
1
f (y) = (2π)−n/2|Σ|−1/2 exp − (y − µ)0Σ−1(y − µ)
2
where |Σ| is the determinant of Σ.
Compare with the univariate normal pdf N (µ, σ 2):
1
−1/2
−1
f (y) = (2π)
σ exp − (y − µ)2/σ 2 .
2
The quadratic term (y − µ)0Σ−1(y − µ) is “a multivariate analog
for the standardized distance of the vector y from the mean
vector µ.”
7
Properties:
• Linear combinations are normally distributed: l1Y1 + l2Y2 +
· · · + lnYn = l0Y is N (l0µ, l0Σl);
• Conditional distributions are normal: given Yj 0 = yj 0 , j 0 6= j,
Yj is normal with:
– a mean that is a linear function of yj 0 , j 0 6= j;
– a variance that does not depend on yj 0 , j 0 6= j.
8
Notes:
• Multivariate normality is a strong assumption.
• The assumption is not critical for validity of inferences about
β.
• Assumptions about dependence, i.e. the structure of Σ, are
more important.
• The multivariate normal distribution is a convenient approximation for continuously distributed responses.
9
Pairwise scatter plots, Succimer group
10
40
●
●
● ●
● ● ●
●●
●●
●
●
●
● ●●●
●● ●
●●
●
● ●●
●
●
●● ●●
●
●
● ● ●●● ●
●
●●
●●
●
●●
●
●●
●
●
y1
●
●
●
●●
● ● ●●
●
●●
● ●
●
●
●
● ●
●●
●
●
● ●●●
●
●●
●●
● ●● ● ●●
●
●
●
●
●● ●
●●
●
●●
●
●●
● ●● ● ●●●
●
●
● ● ●
●
●
●
●●●
●
●
● ●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●● ●
●
●
●
● ● ● ●●●
●
●
●
● ●
● ●●
●
● ●●●
● ● ●● ●
●
●
●●
●
●● ●
●●
●●
●
●
●
●
●
y4
●
●
● ●
●● ●●
●●
●
●●
●●●
●●
●●
●
●
●●●
● ●●●●
●
●
●●● ●
●
● ●●
●
●
●
●●
●
●
●
50
●
●
●
●
●
●
●
●●
● ● ●
●●
●
●
●
●●
●●
● ●● ●
● ●●
●
●●●●
●
●●
●
●
●●●
●●
●
●
● ●● ●
●●
●
●
●
●
●
● ●● ●
● ● ●● ●●●
●●
●● ● ●
● ●
●
●
●
●
●● ●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
40
●●
●
●
●●
●
●
●
●
●
● ●●
●
●● ●
● ●●● ●
●
●●● ● ●●
●●
●●●●
●
● ●
●
●
●●
● ●● ●
●
●
●
●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●●
●
● ●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●● ●
●
●
30
20
10
●
30
y0
●
●
●
●
25
●
● ●
●
20
●
50
40
●
30
35
40
30
30
20
20
10
10
10
30
●
●
●●
●
●
● ●
● ●●
●●
● ● ●●
●●
●
●
●
●
●
●
●
●●
● ●●● ●
● ●●
●
●● ● ● ●
●●
●
●
●
●
20
25
30
●
●●
●
●● ● ● ●
●
● ●●●
●
●
●
●● ●
●
●
●
●
●●
●
●
●●
●● ●
● ●
●
● ●
●● ●
●
●
●
35
40
●
●
●
●
●●
● ●
●
● ●
●● ●
●
●
●
●
●
● ●●●
●●●
● ●●●
●●●
●●
●●
●
●
● ● ●●●
●●
●
●
10
20
30
y6
40
10
Pairwise scatter plots, Placebo group
y0
35
●
25
15
●●
35
25
15
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●●
●
●
● ●●●●● ●
●●●● ●
●
●
●
●
●
●
30
35
●
●
●
●
●
●
●
●
●
●●
● ●
● ●
●
●●
●
●
●●
● ● ●●
●●
● ●●●
● ●
●●
●
●●
●
●
●●●
●● ●
●
●● ●
●
●
●
●
●
●
●● ●
●
●● ●
●
●●● ●
●●
●
●
●
●
●
●
●
●● ● ●
●
●
●●
●●●
●
● ● ●
● ●
y4
●
●
●
● ● ●
●
●● ●●
● ●●
●
● ●●
● ●
●●
●
●
● ● ●●
●●
●●
●●
● ●●
●
● ●●●●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
● ●
●
●
● ●●
●
●●
●●●
●
●● ●● ● ●
●
●●
●
● ●
●
35
● ●
●
●
●
●
●
●
●
●
● ●●
●
● ● ● ●
●●● ●
● ●
●
●
●
●
●●
●
●
●●
●●
●
●
●●● ● ●●●
●
●●
●
●
25
●
●
●
●
●
● ●●
●
● ●
●
● ●
● ●●
●
●
●●● ● ●
● ●●
●
● ●●●●
●●●● ●
●●
● ●● ●
●●●
●
●
y1
●
● ●
●
●
●●
●
●
●●
● ●● ●
●
●
●
●
●
●
●● ●●●● ●
●
●●
●
●●
●●
●
●
●
●
●
●
20
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●●
●
●
●
●
● ●
●●
● ● ●● ●
● ● ● ●●
●
●●
●
●
●
●
●
●
●● ●
● ●
35
●
●
●
● ● ●
● ●
● ● ●● ●
● ● ● ●
●
●
●
●
●●
●
●
●●
● ●
●● ●
●●●● ●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
25
30
●
●
●
●
● ● ●
● ●●
●
● ● ●
●
●
● ●● ●
●●
●● ●
●
● ●
●
●
● ●
●
● ●●
●
● ●●● ●
●
● ●● ●
15
25
35
20
25
●
●
15 20 25 30 35
15
●
●
●
● ● ● ● ●● ●
●
●
●
●
● ●●
●
●●●
●
●
●● ●
●
●
●
●●●
●
●
●
●● ●●
●
●
●
●●
●
●●
●
y6
15 20 25 30 35
11
Succimer group
−1
0
1
40
30
●
20
●
●
10
Sample Quantiles
40
35
30
●
●●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
● ●●●●
−2
●
2
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
● ●●●●
−2
−1
0
1
2
Theoretical Quantiles
Normal Q−Q Plot: y4
Normal Q−Q Plot: y6
40
Theoretical Quantiles
●
−2
−1
0
1
Theoretical Quantiles
50
●
30
●
10
●●
●●●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●
●●
● ●●
●
Sample Quantiles
20
30
●
10
Sample Quantiles
Normal Q−Q Plot: y1
●
25
20
Sample Quantiles
Normal Q−Q Plot: y0
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●●●●●
●
2
−2
−1
0
1
2
Theoretical Quantiles
12
Placebo group
Normal Q−Q Plot: y0
−2
−1
0
1
35
25
−2
−1
0
1
2
Theoretical Quantiles
Normal Q−Q Plot: y4
Normal Q−Q Plot: y6
●
−2
0
1
Theoretical Quantiles
2
35
●
25
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●●
●●
−1
●
Sample Quantiles
15 20 25 30 35
Sample Quantiles
●
Theoretical Quantiles
●
●●
●●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
● ●●●
2
●
●
●
15
●
●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●
●
●
●●
15
●
●
Sample Quantiles
35
30
25
20
Sample Quantiles
●
Normal Q−Q Plot: y1
●
●
●●
●●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●●
−2
−1
0
1
●
2
Theoretical Quantiles
13