MULTILEVEL MODELS
Multilevel-analysis in SPSS - step by step
Dimitri Mortelmans
Centre for Longitudinal and Life Course Studies (CLLS)
University of Antwerp
Overview of a strategy
1. Data preparation (centering and standardizing)
2. Null random intercept model
3. Random intercept with Level-1 predictors
4. Adding Level-2 predictors to step 3
5.
6.
7.
8.
Testing random slopes level 1
Testing significant random slopes Level 1
Testing random slopes with level-2 predictors
Reducing parameters in var-covar
9. Testing cross-level interactions
10. Change estimation method and compare models
11. Standardizing
2
1
The basic formula
• General multilevel-model
Yij=
00 + 10 Xij + 01 Zj + 11 Zj Xij + u1j Xij + 0j + rij
All fixed parameters are in this part
All random parameters are in this part
= FIXED PART
= RANDOM PART
3
Overview of ML models
Model
0RI
Null random intercept / Intercept only /
Unconditional means
RI
Random intercept
FR
Fully Random
4
2
PHASE 1: PREPARATION
1. Data preparation
STEP 1 Data preparation
Actions:
• Look at the distribution of variables
• Check outliers
• Data centering and standardizing
• Check your centering and standardizing
(see slides data management)
6
3
PHASE 2: VARIANCE
COMPONENT MODELS
2. Null random intercept model
3. Random intercept with Level-1 predictors
4. Adding Level-2 predictors to step 3
PHASE 2
2. Null random intercept model
• PURPOSE: estimate a basic model to compare
other models with
3. Random intercept with level-1 variables
• PURPOSE: how much variance can be explained
with your level-1 predictors ?
4. Random intercept with level 1 and level 2
variables
• PURPOSE: how much (additional) variance can
be explained with your level-2 predictors ?
8
4
STEP 2 Null random intercept model
THEORETICAL: Yij= 00 + 0j + rij
In the example:
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
9
STEP 2 Null random intercept model
* 2. STEP 2: Null random intercept model;
* **********************************************;
MIXED ink_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT
/RANDOM = INTERCEPT | SUBJECT(cntry2).
10
5
STEP 2 Null random intercept model
* 2. STEP 2: Null random intercept model;
* **********************************************;
MIXED ink_centr
Put the Dependent
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT
/RANDOM = INTERCEPT | SUBJECT(cntry2).
variable after MIXED
11
STEP 2 Null random intercept model
* * 2. STEP 2: Null random intercept model;
* **********************************************;
MIXED ink_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
PRINT
/FIXED = INTERCEPT
SOLUTION = Show the Fixed effects in the output
/RANDOM = INTERCEPT | SUBJECT(cntry2). with the standard errors
TESTCOV = Show significance tests for the
variance components
METHOD The estimation method
Used method here = Restricted Maximum Likelihood
12
6
STEP 2 Null random intercept model
* * 2. STEP 2: Null random intercept model;
* **********************************************;
MIXED ink_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
FIXED
/FIXED = INTERCEPT
Add the fixed variables (independent variables) here.
/RANDOM = INTERCEPT | SUBJECT(cntry2).
RANDOM
Add the random parts here.
HERE we add only the intercept (for now).
13
STEP 2 Null random intercept model
OUTPUT FOR: Yij= 00 + 0j + rij
Grouping variable
Number of parameters in the model
14
7
STEP 2 Null random intercept model
OUTPUT FOR: Yij= 00 + 0j + rij with
 
rij ~ N 0,r2
0j ~ N0, :   20 
Estimated mean income
over all countries is -0.097
Unexplained variance (differences) on
individuea level equals 4.0058
Unexplained variance (differences) on
Country level (level 2) equals 2.4855
15
STEP 2 Null random intercept model
• Summary table
0RI
Intercept
-0.097596
σ²r
4.005854
σ²0
2.485587
σ²1
//
16
8
STEP 2 Null random intercept model
• No explanation of the variance in income
• BUT a decomposition of the variance in income
in 2 parts
σ²0 = Inter-group variance: differences between countries
σ²r
= Intra-group variance: differences between individuals
QUESTION: Are there enough differences between countries to
justify a multi-level analysis ?
17
STEP 2 Null random intercept model
QUESTION: Are there enough differences between countries to
justify a multi-level analysis ?
ANSWER: Intra-class correlation-coefficient (RHO)
1
= σ²0 / (σ²0 + σ²r)
18
9
STEP 2 Null random intercept model
Intra-class correlation-coefficient (RHO) (1
1
=
σ²0 / (σ²0 + σ²r) )
= 2.4842 / (2.4842 + 4.0076) = 0.383
INTERPRETATION: 38.3 % of the total variance can be ascribed
to level 2.
PUT DIFFERENTLY: For 38.3 %, the differences in income
between countries in the ESS are due to differences between
countries (richer and poorer countries). 61.7 % is due to
differences in individuals within these countries (richer and
poorer individuals within a country).
19
STEP 3 Random intercept (level 1)
THEORETICAL: Yij= 00 + 10 Xij + 0j + rij
In the example:
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
QUESTION: How much of the original variance in
0j has been explained now ?
20
10
STEP 3 Random intercept (level 1)
BASIC IDEA:
Add all your level-1 predictors
De variance components in the residuals should
become smaller BECAUSE the level-1
predictors now take up a part of the
explanation of the total variance.
QUESTION: How big is the explained part ?
21
STEP 3 Random intercept (level 1)
* 3. STEP 3: Random intercept model with level 1 predictors;
WITH: treat the independent as a continuous variable
* **********************************************************************;
BY: treat the independent as a categorical variable
MIXED ink_centr BY geslacht WITH opl_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr
/RANDOM = INTERCEPT | SUBJECT(cntry2).
Add your Level-1 independent variables
22
11
STEP 3 Random intercept (level 1)
OUTPUT FOR:
Yij=
00 + 10 Xij + 0j + rij
 
rij ~ N 0,r2
0j ~ N0,  :   20 
Overall intercept
Coefficients at level-1
Error-variance on level 1
Error-variance on level 2
23
STEP 3 Random intercept (level 1)
OUTPUT FOR:
Yij=
00 + 10 Xij + 0j + rij
 
rij ~ N 0,r2
0j ~ N0,  :   20 
The effect of gender and
education on income is
significant.
Even after controlling for level-1 predictors
there still are significant differences between individuals
on level-1 and significant differences between the countries
on level 2.
24
12
STEP 3 Random intercept (level 1)
Reduction of individual differences:
From 4.0058 to 3.4640
Reduction of differences between countries:
From 2.4855 to 2.1303
Is this enough ? We need an R²-measure
25
STEP 3 Random intercept (level 1)
Regression-like R²-measure on level 1:
rBasis - rModel
/ rBasis
4.0058 - 3.4640 / 4.0058
Regression-like R²-measure on level 2:
Basis - Model / Basis
2.4855 - 2.1303 / 2.4855
BUT: The estimation methods make this easy solution erroneous.
26
13
STEP 3 Random intercept (level 1)
No consensus yet about a good R²-measure.
Correction of Bosker&Snijders (1994) – level 1:
(rBasis + Basis) - (r
Model
+
Model)
/ (rBasis + Basis)
Correction of Bosker&Snijders (1994) – level 2:
(rBasis + (Basis /n) )-(r
Model
+ (
Model/n)
) / (rBasis + (Basis/n) )
Whereby n = mean cluster size at level 2
27
STEP 3 Random intercept (level 1)
Correction of Bosker&Snijders (1994) – level 1:
(rBasis + Basis) - (r
Model
+
Model)
/ (rBasis + Basis)
= 0.14
Correction of Bosker&Snijders (1994) – level 2:
(rBasis + (Basis /n) )-(r
Model
+ (
Model/n)
) / (rBasis + (Basis/n) )
= 0.14
28
14
STEP 4 Random intercept (level 1+2)
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 0j + rij
In the example:
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
01 Zj
= Direct effect of country variables on income (BNP or religion)
QUESTION: How much variance in
0j
from step 3 is now further explained ?
29
STEP 4 Random intercept (level 1+2)
BASIC IDEA:
Look how much the level 2 predictors can
explain (as a group)
30
15
STEP 4 Random intercept (level 1+2)
* 4. STEP 4: Random intercept model with level 1 and level 2 predictors;
* *********************************************************;
MIXED ink_centr BY geslacht WITH opl_centr bnp_centr bnppps_centr romkath_centr relig_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr bnp_centr bnppps_centr romkath_centr relig_centr
/RANDOM = INTERCEPT | SUBJECT(cntry2).
31
STEP 4 Random intercept (level 1+2)
OUTPUT FOR:
Yij=
00 + 10 Xij + 01 Zj + 0j + rij
 
rij ~ N 0,r2
0j ~ N0, :   20 
Overall intercept
Coefficients on level-1
Coefficients on level-2
Error-variance on level 1
Error-variance on level 2
32
16
STEP 4 Random intercept (level 1+2)
OUTPUT FOR:
Yij=
00 + 10 Xij + 01 Zj + 0j + rij
 
rij ~ N 0,r2
0j ~ N0, :   20 
BNPPPS and % Roman Catholics are
significant,
BNP and Religiosity not.
Even after controlling for level-2 predictors
there still are significant differences between individuals
on level-1 and significant differences between the countries
on level 2.
33
STEP 4 Random intercept (level 1+2)
OUTPUT FOR:
Yij=
00 + 10 Xij + 01 Zj + 0j + rij
 
rij ~ N 0,r2
0j ~ N0, :   20 
Problem with BNP ?
34
17
STEP 4 Random intercept (level 1+2)
Reduction of individual differences:
3 vs 2
3 vs 1
From 3.464 to 3.476
From 4.006 to 3.476
Reduction of country differences:
3 vs 2
3 vs 1
From 2.130 to 0.552
From 2.486 to 0.552
35
STEP 4 Random intercept (level 1+2)
Correction of Bosker&Snijders (1994) – level 1:
Pseudo – R² = 0.38
Correction of Bosker&Snijders (1994) – level 2:
Pseudo – R² =
0.78
36
18
STEP 4 Random intercept (level 1+2)
Preliminary conclusions:
1. The error-variance on level 2 is only significant to the
0.006 level.
2. Two level-2 variables do not seem to work in this
model: BNP and Religiosity.
We remove them from the model when they are less important from a theoretical
perspective.
3. Our model strongly explains differences on level 2. But
also on the individual level the explanatory power of
the independent variables is satisfactory.
37
PHASE 3: testing random
slopes
5.
6.
7.
8.
Testing random slopes level 1
Testing significant random slopes Level 1
Testing random slopes with level-2 predictors
Reducing the parameters in var-covar
19
PHASE 3
5. Testing random slopes level 1
• PURPOSE: looking which level-1 predictor has
significant slopes (without the level-2 predictors).
6. Testing significant random slopes Level 1
• PURPOSE: joining all significant random slopes from
step 5 in one model
7. Testing random slopes with level-2 predictors
• PURPOSE: adding the level 2 predictors back to
model, including the random slopes from step 6
8. Reducing parameters in var-covar
• PURPOSE: simplifying the model (removing nonsignificant co-variances) in order to ease the
estimation of the model.
39
STEP 5 Testing random slopes level 1
THEORETICAL : Yij= 00 + 10 Xij + 1j Xij + 0j + rij
In the example:
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
1j Xij
= Differences in slopes (effect) of variable X (gender or education)
between the countries
REMARK: No longer country differences in this model !
REMARK : Model is estimated for each independent variable on level 1 separately
QUESTION: Are there significant differences in the effect of X between the countries?
40
20
STEP 5 Testing random slopes level 1
BASIC IDEA:
Until now we did not allow for differences in
slopes of the independent variables at level 1.
FOR EACH INDEPENDENT, we will test if there
are significant differences in slopes at level-2.
41
STEP 5 Testing random slopes level 1
* STEP 5: Testing the significance of the random slopes step-by-step.
***********************************************************************************;
MIXED ink_centr BY geslacht WITH opl_centr
COVTYPE should be “Unstructured”.
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr
/RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2) COVTYPE(UN).
We make eduction RANDOM here.
42
21
STEP 5 Testing random slopes level 1
OUTPUT FOR: Yij=
00 + 10 Xij + 1j Xij + 0j + rij
 
rij ~ N 0,r2
 20

0j
  ~ N0,  :    2
2 
 1j
01 1
Overall intercept
Coefficients on level-1
Error-variance on level 1
Error-variance on level 2
43
STEP 5 Testing random slopes level 1
Structure of the variance-covariance matrix
Theoretical:
In SPSS:
 20

0j



~
N

0,

:
 

  2
2 
 1j
01 1
UN1,1

UN2,1 UN2,2


 2.1123

0.03258 0.01991



44
22
STEP 5 Testing random slopes level 1
Structure of the variance-covariance matrix
In SPSS:
UN1,1

UN2,1 UN2,2


 2.1123

0.03258 0.01991



The random slope is significant on the 0.01 level.
MEANING: there is a significant different effect of education between the countries
45
STEP 5 Testing random slopes level 1
* STEP 5: Testing the significance of the random slopes step-by-step.
**********************************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr
/RANDOM = INTERCEPT geslacht | SUBJECT(cntry2) COVTYPE(UN).
Gender is made RANDOM here.
46
23
STEP 5 Testing random slopes level 1
OUTPUT FOR: Yij=
00 + 10 Xij + 1j Xij + 0j + rij
 
rij ~ N 0,r2
 20

0j
  ~ N0,  :    2
2 
 1j
01 1
Overall intercept
Coefficients on level-1
Error-variance on level 1
Error-variance on level 2
47
STEP 5 Testing random slopes level 1
Structure of the variance-covariance matrix
Theoretical:
In SPSS:
 20

0j
~
N

0,

:



 

  2
2 
 1j
01 1
UN1,1

UN2,1 UN2,2


 1.144

0.4728 0.6207


48
24
STEP 5 Testing random slopes level 1
Evaluation of the random
slope of gender
In SPSS:
  2.1725
UN1,1

UN2,1 UN2,2 

  0.04622 0.0097

The random slope of the dichotomous variable gender can not be tested.
If you include “gender” as a continuous variable in the model, this random slope
turns out to be significant.
49
STEP 5 Testing random slopes level 1
Conclusion:
- Education and gender have significant slopes between
the countries.
- Gender is only estimable when you include the variable
as a continuous measure.
BUT: the effect of gender is unclear.
In this example I will keep gender in the model with a
random slope (only for educational purposes).
50
25
STEP 6 Testing random slopes level 1+2
THEORETICAL : Yij= 00 + 10 Xij + 1j Xij + 0j + rij
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
1j Xij
= Differences in slopes (effect) of variable X (gender or education)
between the countries
Theoretically this is the same model as in step 5 but now we include
ALL significant random slopes in the model.
51
STEP 6 Testing random slopes level 1+2
* STEP 6: Including all level-1 variables with significant random slopes.
* ****************************************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr
/RANDOM = INTERCEPT opl_centr geslacht | SUBJECT(cntry2) COVTYPE(UN).
52
26
STEP 6 Testing random slopes level 1+2
OUTPUT FOR: Yij=
00 + 10 Xij + 1j Xij + 0j + rij
 
rij ~ N 0,r2
 20

0j
  ~ N0,  :    2
2 
 1j
01 1
Error-variance on level 1
Error-variance on level 2
Equal digits are error-variances:
UN(1,1)
Intercept
UN(2,2)
Error-variance of education
UN (3,3)
Error-variance of gender
53
STEP 6 Testing random slopes level 1+2
OUTPUT FOR: Yij=
00 + 10 Xij + 1j Xij + 0j + rij
 
rij ~ N 0,r2
 20

0j
  ~ N0,  :    2
2 
 1j
01 1
Equal digits are error-variances:
UN(1,1)
Intercept
UN(2,2)
Error-variance of education
Significant on the 0.01 level
UN (3,3)
Error-variance of gender
NOT significant:
SO: Gender NOT random in the model!
54
27
STEP 7 Testing random slopes with level-2 predictors
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
1j Xij
01 Zj
= Differences in slopes (effect) of variable X (gender or education)
between the countries
= Direct effect of country variables on income (BNP or religion)
55
STEP 7 Testing random slopes with level-2 predictors
BASIC IDEA:
We now check which influence the level-2
predictors have on income in a model with
random slopes (= step 6).
56
28
STEP 7 Testing random slopes with level-2 predictors
* STEP 7: Including all level-2 variables in the model of step 6
*. ******************************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr
/RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2) COVTYPE(UN).
57
STEP 7 Testing random slopes with level-2 predictors
OUTPUT FOR:
Yij=
00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
 
rij ~ N 0,r2
 20

0j
  ~ N0,  :    2
2 
 1j 
01 1
Overall intercept
Coefficients on level-1
Coefficients on level-2
Error-variance on level 1
Error-variance on level 2
58
29
STEP 7 Testing random slopes with level-2 predictors
OUTPUT FOR:
Yij=
00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
 
rij ~ N 0,r2
 20

0j
  ~ N0,  :    2
2 
 1j 
01 1
Both the predictors on level 1 as on
level 2 are significant.
Significant differences remain
between the intercepts of the
countries.
Significant differences remain on
level 2 for the effect of education
59
STEP 8 Simplifying the model
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
1j Xij
01 Zj
= Differences in slopes (effect) of variable X (gender or education)
between the countries
= Direct effect of country variables on income (BNP or religion)
We now look at the variance-covariance matrix
60
30
STEP 8 Simplifying the model
Differences in intercept and slope between units of level 2.
Yij=
00 + 10 Xij + 1j Xij + 0j + rij
These are the conditions:
This is the covariance between the
random intercepts and the random
slopes for variabele Xij.
rij ~ N(0, σ²r)
 20

0j


~
N
0,
:



 

  2
2 
 1j
01 1
If this covariance is significant (=
different from 0), then it means that the
changes in intercepts is systematically
correlated to the changes in slopes.
61
STEP 8 Simplifying the model
Model
intercept
Slopes
Covariance
FR
many
many
Negative
FR
many
many
Positive
FR
many
many
Not
significant
(0)
62
31
STEP 8 Simplifying the model
With 1 independent X-variable on level 1
 20

0j


~
N
0,
:



 

  2
2 
 1j
01 1
Covariance between the random
intercepts and the random slopes for
variabele Xij.
With 2 independent X-variables on level 1

 20
0j

 2
 
2

1j ~ N0,  :   01 1
202 212 22 
2j
 


2 independents = 3 covariances
63
STEP 8 Simplifying the model
With 3 independent X-variable on level 1
 20

0j
 2

 
2

 1j ~ N0,  :   01 1


202 212 22

2j
 2
 
2
2
2 
03 13 23 3
3j
3 independents = 6 covariances
With random slopes models, the number of parameters to be
estimated increases very rapidly.
64
32
STEP 8 Simplifying the model
With 3 independent X-variables on level 1
 20

0j
 2

 
2

 1j ~ N0,  :   01 1


202 212 22

2j
 2
 
2
2
2 
3j
03 13 23 3
Non-significant covariances = The covariances is not
significantly different from 0.
SO: We might as well fix these to 0.
CONSEQUENCE: Each fixed covariance is one
parameter less to be estimated.
65
STEP 8 Simplifying the model
With 3 independent X-variables on level 1
 20

0j
 2

 
2

 1j ~ N0,  :   01 1


202 212 22

2j
 2
 
2
2
2 
03 13 23 3
3j
SPSS does not allow to fix each separate covariance to 0.
BUT: there is an option to fix all co-variances to 0 in one
time.
66
33
STEP 8 Simplifying the model
With 3 independent X-variables on level 1
 20

0j
 2

 
2

 1j ~ N0,  :   01 1


202 212 22

2j
 2
 
2
2
2 
3j
03 13 23 3
20

0j


 
2

 1j ~ N0,  :    0 1


 0 0 22

2j

 
2 
3j
 0 0 0 3
COVTYPE(UN)
Without COVTYPE
67
STEP 8 Simplifying the model
Look at the (significance of) the co-variances in
the output of step 7
Covariance is NOT significant
Consequence: Estimate the model in STEP 7 again without the COVTYPE-option.
68
34
STEP 8 Simplifying the model
•STEP 8: Reduction of parameters in var-covar matrix
•*********************************************************.
MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr
/RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2).
69
STEP 8 Simplifying the model
The co-variances are fixed at 0 : there is no estimation of co-variances any more.
Only the variances are given in the output.
70
35
PHASE 4: Refining and
finishing
9. Testing cross-level interactions
10. Change the estimation method and compare your
models
11. Standardizing
PHASE 4
9. Testing cross-level interactions
• PURPOSE: checking if level-2 predictors correlate
with level-1 predictors
10.Changing the estimation method and
comparing models
• PURPOSE: Compare models with Full Maximum
Likelihood
11.Standardizing
• PURPOSE: Estimating the end model again with
standardized variables to check the relative
influence
72
36
STEP 9 Testing cross-level interactions
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 01 Zj Xij + 1j Xij + 0j + rij
00
= Overall intercept: mean income over all countries
0j
= Non explained differences in income between the countries
rij
= Non explained differences in income between the individuals
10 Xij
= General slope of the independent variable X (gender or education)
1j Xij
01 Zj
= Differences in slopes (effect) of variable X (gender or education)
between the countries
= Direct effect of country variables on income (BNP or religion)
01 Zj Xij = Interaction of country variables and individual variables on income
73
STEP 9 Testing cross-level interactions
What is an interaction effect ?
A (significant) interaction effect shows how the
effect of one independent variable changes
within the levels of another independent
variable.
74
37
STEP 9 Testing cross-level interactions
• In regular OLS regression
EG. Interaction between gender and education:
How does the effect of education on income change between
boys and girls?
• In multilevel regression
EG. Interaction between education and GDP:
How does the effect of education on income change for
countries with a different GDP ?
75
STEP 9 Testing cross-level interactions
TWO BASIC PRINCIPLES
1.When an interaction effect is significant, the
main effects NEED TO BE included in the model.
EG:
When the interaction effect between education and
GDP_PPS is included
THEN you also need the main effect of education and the
main effect of GDP_PPS in the model.
76
38
STEP 9 Testing cross-level interactions
TWO BASIC PRINCIPLES
2. When an interaction effect is present in a model, the
main effects have a different meaning.
Meaning without interaction-effect
- Education: the change in income when the educational
level rises with one unit
-
BNP_PPS: the change in income when BNP_PPS rises
with one unit
77
STEP 9 Testing cross-level interactions
TWO BASIC PRINCIPLES
2. When an interaction effect is present in a model, the
main effects have a different meaning.
Meaning without interaction-effect
- Education: the change in income when the educational
level rises with one unit at a level of 0 from BNP_PPS
-
BNP_PPS: the change in income when BNP_PPS rises
with one unit at a level of 0 from education
78
39
STEP 9 Testing cross-level interactions
* * STEP 9: Testing cross-level interactions
* **********************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr
/PRINT = SOLUTION TESTCOV
/METHOD = REML
/FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr
opl_centr*bnppps_centr
/RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2).
Specifying an interaction effect = repeating both variables with * between both.
79
STEP 9 Testing cross-level interactions
Main effects (both are significant)
Interaction effect (not significant)
80
40
STEP 9 Testing cross-level interactions
How to interpret the interaction effect ?
(if it had been significant)
1. Interpret the main effects
2. Choose an interpretation perspective
3. Write the regression equation for one of the
independent variables
81
STEP 9 Testing cross-level interactions
Interpretation 1. Interpret the main effects
Main effect education: When BNPPPS_centr is equal to 0, someone's income will increase with 0.5192
scale points when his educational level increases with one unit.
Main effect GDP: When education is equal to 0, someone's income will increase with 0.0265 scale
points when his countries BNP_PPS increases with one unit.
82
41
STEP 9 Testing cross-level interactions
Interpretation 1. Interpret the main effects
Because of the centering (mean = 0) we can also say:
Main effect education: At a mean BNPPPS_centr level, someone's income will increase with 0.5192
scale points when his educational level increases with one unit.
Main effect GDP: At a mean educational level, someone's income will increase with 0.0265 scale
points when his countries BNP_PPS increases with one unit.
83
STEP 9 Testing cross-level interactions
Interpretation 2. Choose an interpretation perspective
2 perspectives:
1. How does the effect of education on income changes
for different levels of BNP_PPS ?
2. How does the effect of BNP_PPS on income changes
for different levels of education ?
84
42
STEP 9 Testing cross-level interactions
Interpretation 2. Choose an interpretation perspective
“How does the effect of education on income changes for
different levels of BNP_PPS ?”
is theoretically the only possibility here.
BECAUSE: BNP_PPS will not change if people have a
higher degree. But the educational effects can indeed
change for countries with different BNP_PPS.
BUT REMEMBER: Statistically, both perspectives are
always possible.
85
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
PERSPECTIVE: How does the effect of education on
income changes for different levels of BNP_PPS ?
-
Write out the regression equation for education, when
BNP_PPS takes on different levels.
-
WHICH levels of BNP_PPS ?
Usually: minimum, 25th percentile, median, mean, 75th
percentile and maximum.
86
43
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
STEP 1: Search in the EXAMINE output for the
values of the minimum, 25th percentile, median,
mean, 75th percentile and the maximum of
BNPPPS_centr.
EXAMINE VARIABLES=bnppps_centr
/PERCENTILES(25,75)
/STATISTICS DESCRIPTIVES.
87
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
STEP 1: Search in the EXAMINE output for the
values of the minimum, 25th percentile, median,
mean, 75th percentile and the maximum of
BNPPPS_centr.
88
44
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
STEP 1
Minimum
Q1
Mean
Median
Q3
maximum
-65.8815
-23.2815
0
2.1185
14.6185
126.9125
Remark: We assume that the other independent variable also
take on the value of 0 (and disappear from the equation).
89
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
Income = -0.2532 + 0.5192 education + 0.0265 BNPPPS + 0.0008 EDUC*BNPPPS
STEP 2: Vul in
Minimum
Q1
Median
Mean
Q3
maximum
Income
Income
Income
Income
Income
Income
=
=
=
=
=
=
-0.2532
-0.2532
-0.2532
-0.2532
-0.2532
-0.2532
+
+
+
+
+
+
0.5192
0.5192
0.5192
0.5192
0.5192
0.5192
opleiding
opleiding
opleiding
opleiding
opleiding
opleiding
+
+
+
+
+
+
0.0265
0.0265
0.0265
0.0265
0.0265
0.0265
(-65.88) + 0.0008 OPL*(-65.88)
(-23.2815) + 0.0008 OPL*(-23.2815)
(0) + 0.0008 OPL*(0)
(2.1185) + 0.0008 OPL*(2.1185)
(14.6185) + 0.0008 OPL*(14.6185)
(126.92) + 0.0008 OPL*(126.92)
90
45
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
Income = -0.2532 + 0.5192 education+ 0.0265 BNPPPS + 0.0008 EDUC*BNPPPS
STEP 2: Vul in
Minimum
Q1
Median
Mean
Q3
maximum
Income
Income
Income
Income
Income
Income
=
=
=
=
=
=
-1,999 + 0,4665 OPL
-0,8702 + 0,5006 OPL
-0,1971 + 0,5209 OPL
-0,2532 + 0,5192 OPL
0,1342 + 0,5309 OPL
3,11
+ 0,6207 OPL
Cfr main effect of education when BNP_PPS is
equal to 0 (see the main effect in the output).
Interpretation interaction effect
The effect of education rises (very slightly) as BNP_PPS increases.
91
STEP 10 Comparing models with Full Maximum Likelihood
• Estimation in multilevel analysis
- Maximum Likelihood
-
Generalized Least Squares (not in SPSS)
Generalized Estimation Equations (not in SPSS)
Markov Chain Monte Carlo (not in SPSS)
Bootstrapping (not in SPSS)
92
46
STEP 10 Comparing models with Full Maximum Likelihood
• Maximum Likelihood
- Maximizing the likelihood function
- Population parameters are estimated in such a way
that the probability of finding that particular model
is maximized with the data at hand.
- TWO versions:
Full Maximum Likelihood
(ML)
Restricted Maximum Likelihood (REML)
93
STEP 10 Comparing models with Full Maximum Likelihood
• Restricted Maximum Likelihood
- Only the variance components are used in the
estimation.
- Standard in SAS (but not in SPSS) and in almost
all multilevel programs
- Best estimation for the variance components
- CONSEQUENCE: Gives you the best estimation for
the parameters and you should use this
estimation method to obtain your parameters
94
47
STEP 10 Comparing models with Full Maximum Likelihood
• Full Maximum Likelihood
- Variance components and regression coefficients are
involved in the estimation.
- Disadvantage: Variance components are
underestimated.
USEFULL ?
- Easier estimation method
- The difference between two models can be used in a
chi²-difference test (is not possible with REML !).
95
STEP 10 Comparing models with Full Maximum Likelihood
• CONSEQUENCE
- Use REML for all models
- Use ML at the end to compare estimated models with
each other.
96
48
STEP 10 Comparing models with Full Maximum Likelihood
* STEP 10: Using Full Maximum Likelihood to compare models.
* ************************************************************************************.
*Step2.
MIXED ink_centr
/PRINT = SOLUTION TESTCOV Use Method=ML (or leave the METHOD line out).
/METHOD = ML
/FIXED = INTERCEPT
/RANDOM = INTERCEPT | SUBJECT(cntry2).
97
STEP 10 Comparing models with Full Maximum Likelihood
Use the difference in DEVIANCE (-2 LOG
LIKELIHOOD) and the difference in degrees of
freedom in a Chi²-difference test
98
49
STEP 10 Comparing models with Full Maximum Likelihood
• Calculating the degrees of freedom of your model
Calculating the degrees of freedom by hand:
Intercept
1
Error variance level-1
1
Fixed slopes on level-1
p
Error-variances for slopes on level-1
p
Co-variances of slopes and intercept
p
Co-variances between the slopes
p(p-1)/2
Fixed slopes on level-2
q
Slopes for cross-level interactions
p*q
Whereby:
p = number of level-1 predictors
q = number of level-2 predictors
99
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE
D(f)
Prob. Chi² vs null
model
Prob. Chi² vs
previous model
STEP 2
Null random intercept model
141642,268
2
STEP 3
Random intercept with level-1
136763,937
9
0,000
STEP 4
Random intercept w level 1 and 2
122423,421
13
0,000
STEP 6
Random slopes model
136407,950
9
0,000
STEP 7
Random slopes with level-2
122075,321
11
0,000
0,000
STEP 8
Random slopes with level-2
(reduction)
122077,357
7
0,000
0,729
STEP 9
Cross level interactions
122077,357
12
0,000
1,000
0,000
100
50
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE
D(f)
Prob. Chi² vs null
model
STEP 2
Null random intercept model
141642,268
2
STEP 3
Random intercept with level-1
136763,937
9
0,000
STEP 4
Random intercept w level 1 and 2
122423,421
13
0,000
This is the basic model.
STEP 6
Random slopes model
Random slopes with level-2
0,000
Chi²-difference test
136407,950
9
0,000
- Test value = 141642.268 – 136763.937 = 4878
- Degrees of freedom = 9 – 2 = 7
In the fourth column, you compare with this model
STEP 7
Prob. Chi² vs
previous model
122075,321
11
0,000
0,000
- Result: Probability = 0.000
STEP 8
Random slopes with level-2
(reduction)
122077,357
STEP 9
Cross level interactions
122077,357
- Interpretation:
The model in0,000
STEP3 is a significant
7
0,729
improvement of the Null random intercept model (STEP 2)
12
0,000
1,000
101
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE
D(f)
Prob. Chi² vs null
model
STEP 2
Null random intercept model
141642,268
2
STEP 3
Random intercept with level-1
136763,937
9
0,000
STEP 4
Random intercept w level 1 and 2
122423,421
13
0,000
136407,950
intercept
model (STEP 2) 9
0,000
STEP 6 Comparison with
Random
slopes
model
the Null
random
Prob. Chi² vs
previous model
0,000
Interpretation: Is this model a significant improvement vs STEP 2 ?
STEP 7
Random slopes met level-2
122075,321
11
0,000
0,000
Comparison with the previous model (= STEP 3)
STEP 8
Random slopes met level-2
(reductie)
STEP 9
Cross level interacties
122077,357
Interpretation:7
122077,357
12
0,000
Is this model a significant
improvement 0,729
vs STEP 3?
0,000
1,000
102
51
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE
D(f)
Prob. Chi² vs null
model
Prob. Chi² vs
previous model
STEP 2
Null random intercept model
141642,268
2
STEP 3
Random intercept met level-1
136763,937
9
0,000
STEP 4
Random intercept met level 1 en 2
122423,421
13
0,000
STEP 6
Random slopes model
136407,950
9
0,000
STEP 7
Random slopes met level-2
122075,321
11
0,000
0,000
0,000
0,729
0,000
1,000
0,000
BE CAREFULL here !!
STEP 8Previous
Random slopes met level-2
122077,357
7 without comodel is the model
with co-variances.
SO is the model
(reductie)
variances a significant worsening vs. the previous model ?
STEP 9
Cross level interacties
122077,357
12
CONSEQUENCE:
A probability higher than
0.05 points to a good
model.
103
STEP 11 The final model with standardized variables
• Standard output of multilevel models = not
standardized.
• Interpretation in original (scale) units
• DISADVANTAGE: no comparison of the actual
size of the parameters.
104
52
STEP 11 The final model with standardized variables
• 2 methods
1. Estimate the last model again with
standardized variables
(disadvantage = variance components also change)
2. Standardize the parameter by hand
105
STEP 11 The final model with standardized variables
METHOD 1
* STEP 11: Standardising in order to compare parameters. *
********************************************************************************************;
MIXED ink_std BY gesl_std WITH opl_std bnppps_std romkath_std
/PRINT = SOLUTION TESTCOV
/METHOD = ML
/FIXED = INTERCEPT geslacht opl_std bnppps_std romkath_std
/RANDOM = INTERCEPT opl_std | SUBJECT(cntry2).
106
53
STEP 11 The final model with standardized variables
METHOD 2
Stand. Coeff. =
(non stand. Coeff) x (stan dev INdep.)
(stan dev DEpendent)
107
STEP 11 The final model with standardized variables
METHOD 2
DESCRIPTIVES VARIABLES=ink_centr geslacht opl_centr bnppps_centr
romkath_centr /STATISTICS=STDDEV.
108
54
STEP 11 The final model with standardized variables
RESULT:
Non
standardized
Standardized
Gender
0,281
0,06
Education
0,519
0,31
Non
standardized
Standardized
BNP PPS
0,027
0,40
% Rom Cath
-0,095
-0,15
109
MULTILEVEL MODELS
Multilevel-analysis in SPSS - step by step
Dimitri Mortelmans
Centre for Longitudinal and Life Course Studies (CLLS)
University of Antwerp
55