Econometrics II – Heij et al. Chapter 7.7
Panel Data, SUR and GLS
Marius Ooms
Tinbergen Institute Amsterdam
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 1/22
Heij et al. (2004) §7.7.1-7.7.3
• Panel data
• Seemingly unrelated regression model (SUR)
◦ Generalized least squares (GLS)
◦ Feasible GLS
• Panel data with fixed effects
• Panel data with random effects
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 2/22
Panel data
• Panel data consist of cross-section observations for
different time points.
• We observe one dependent variable yit for individual i at
time t where i = 1, . . . , m and t = 1, . . . , n.
• In most applications m is (much) larger than n: m >> n.
• We have k strongly exogenous explanatory variables in a
vector xit and n >> k .
In the literature one often uses N (big N ) for m and T (big T ) for
n. As Heij et al. is mostly time series oriented, they use n as the
time series dimension and m as the cross-section dimension (or
”number of equations” in multivariate time series).
For n = 1 and large m , we have simple cross-section data.
When m = 1 and n is large, we have univariate time-series.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 3/22
General Panel data Model
The considered models are of the following regression form:
yit = αit + x′it γit + εit , Var(ε) = Ω,
where Ω is the mn × mn covariance matrix of the mn × 1
disturbances vector ε and xit is a (k − 1) × 1 vector of
explanatory variables. Parameters depend on time t and on
individual i.
This general model is not empirically identified since it
contains more parameters than observations!
Therefore, we have to impose restrictions on the regression
parameters (αit , γit ) and on the covariance matrix Ω, before we
can estimate parameters.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 4/22
Seemingly unrelated regression model (SUR)
The SUR model is given by
yit = αi + x′it γi + εit
Where E[εit εjt ] = σij ,
E[εit εjs ] = 0 for all i, j and t 6= s.
All individuals have their own regression parameters, but
these are restricted to be constant over time.
The regression relations for the different individuals are only
related via the correlation of the error terms, but the error
covariance across individuals is unrestricted
No error covariance across time: no serial correlation, or serial
cross correlation.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 5/22
SUR model specification and estimation per individual
Denote the observations for unit i by the n × 1 vector yi and by
the n × k matrix Xi , with corresponding parameter vector
βi = (αi , γi′ )′ and n × 1 vector of disturbances εi . The model for
SUR unit i can be written as
yi = Xi βi + εi
Estimating the parameters βi by OLS per equation is
consistent, but is inefficient if the disturbances for the different
individuals display contemporaneous correlation and the
regressor sets differ from individual (equation) to individual
(equation)
This is easily shown if we combine data for all the units in one
big transformed regression model, where we can apply OLS
theory of §3.1.4. See next slides.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 6/22
Complete SUR model in matrix notation
Combining the models for the m units gives






y1
y2
..
.
ym


 
 
=
 
 
X1 0 · · ·
0 X2 · · ·
..
..
.
.
0
0 ···



E[ε] = 0, var(ε) = Ω = 


0
0
..
.
Xm
σ11 I
σ12 I
..
.






σ12 I
σ22 I
..
.
β1
β2
..
.
βm


 
 
+
 
 
ε1
ε2
..
.
εm
···
···
σ1m I
σ2m I
..
.
σ1m I σ2m I · · ·
σmm I












TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 7/22
Inefficiency of OLS for SUR model
Simultaneous OLS estimation of the mk × 1 parameters βi for
i = 1, . . . , m in the above model is equivalent to applying OLS
per unit.
Exercise (1) Prove this proposition. Hint: Consider the
method-of-moments equations for OLS.
This simultaneous OLS estimator is not BLUE since the
covariance matrix Ω is not of the form σ 2 I . Next we will discus a
general method to deal with this problem.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 8/22
Generalized least squares (GLS) idea
First suppose we know Ω up to one constant term.
The idea is to transform the model so that the error covariance
matrix becomes ”scalar” σ 2 I . This idea is similar to WLS.
Transform the joint model by a (big) square and invertible but
nondiagonal weighting matrix A, transform:
y = Xβ + ε
into
Ay = AXβ + Aε
As the variance matrix of Aε is AΩA′ , we choose A s.t.
AΩA′ = I , or A−1 (A′ )−1 = (A′ A)−1 = Ω.
A is a square root of Ω−1 . The decomposition of Ω is standard in
matrix algebra, see section A.6, think of A as A = Ω−1/2 . Note
that A is not uniquely defined.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 9/22
GLS, theoretical infeasible version
Assume the following notation for the transformed variables:
y∗ = Ay, X∗ = AX, ε∗ = Aε,
so that
y∗ = X∗ β + ε∗
with E[ε∗ ] = 0 and Var(ε∗ ) = Inm . Now the BLUE estimator of β
is given by
bGLS = (X∗′ X∗ )−1 X∗′ y∗ = (X ′ Ω−1 X)−1 X ′ Ω−1 y.
This is called the Generalized Least Squares estimator. In
practice this estimator is infeasible as we do not know Ω.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 10/22
Two step Feasible GLS in SUR model
When Ω is unknown, we first estimate Ω with OLS. We then
perform the so-called Feasible GLS in two steps:
• Estimate Ω.
Do m regressions, one per unit to estimate βj by OLS,
j = 1, . . . , m. Let ei be the n × 1 vector of OLS residuals for
unit i. The unknown (co)variances σij are then estimated by
1 Pn
sij = n t=1 eit ejt . Then obtain Ω̂ by replacing σij by sij .
• Estimate the parameters βj jointly by GLS.
That is
bF GLS = (X ′ Ω̂−1 X)−1 X ′ Ω̂−1 y
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 11/22
Asymptotic distribution of FGLS in SUR application
Under the assumption of correct specification and normally
distributed error terms, it can be shown that the FGLS estimator
in the SUR model has the same asymptotic properties as ML,
i.c. for n → ∞ and m fixed one can derive
bF GLS ≈ N (β, (X∗′ X∗ )−1 ) ≈ N (β, (X ′ Ω̂−1 X)−1 ).
We can use this result to perform ’asymptotic’ t- and F -tests. In
practice n is finite and one has to take extra precautions to make
sure X ′ Ω̂−1 X is a full rank positive definite matrix, see also next
slide.
NB: if the number of unknown parameters in Ω increases linearly with n, FGLS does not
work. Compare the GMM standard error derivation for general Ω in §5.5.2
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 12/22
Finite Sample Rank condition SUR, efficiency SUR
The estimated SUR covariance matrix Ω̂ is an mn × mn block
diagonal matrix with the m × m matrix S on the diagonal blocks
with elements sij = n1 e′i ej with ei the n × 1 vector of OLS
residuals of unit i. This means that Ω̂ is invertible, and 2-step
FLGS for SUR possible, if and only if the m × m matrix S is
invertible, i.e. if and only if rank(S ) = m.
Necessary condition for Feasible GLS in SUR: Define the n × m
matrix E = (e1 , · · · , em ) , then S is n1 E ′ E and
m = rank(S) = rank(E) ≤ n. Therefore S and Ω̂ can be
invertible and we can estimate β with FGLS only if m ≤ n.
There are special cases in which OLS is efficient for SUR
models. Exercise (2) 7.10, page 715.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 13/22
Panel data with fixed effects
When m >> n the data are ’typical’ panel data or longitudinal
data. We cannot apply the SUR model, as this requires m ≤ n.
The model has to be simplified by parameter restrictions.
E.g., the coefficients of the explanatory variables are assumed
to be the same for all units (’pooling’): we impose (pooling)
restrictions on the slope parameters γi : γi = γ, i = 1, . . . , m.
We then obtain the panel data model with fixed effects:
yit = αi + x′it γ + εit , εit ∼ IID(0, σ 2 ),
The constant terms αi are fixed unknown parameters, but they
differ from unit to unit (not pooled). The errors are independent
and homoskedastic in time and across units.
NB: the number of parameters increases linearly in m, so
standard asymptotic theory still requires n → ∞, although nearly
all parameters are pooled in the cross section.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 14/22
Fixed effects model in matrix notation I
We can rewrite the model in standard regression form using unit
dummy variables
(
1, i = j
Dit (j) =
, i = 1, . . . , m, j = 1, . . . , m
to get
0, i 6= j
yit =
Pm
j=1 αj Dit (j)
+ x′it γ + εit , εit ∼ IID(0, σ 2 )
Next, define the n × 1 vector yi with elements yit , define εit
accordingly and define the n × (k − 1) matrix Xi with tth row x′it ,
t = 1, . . . , n and let ι be an n × 1 vector of ones.
For the ith unit we obtain the matrix notation
yi = ιαi + Xi γ + εi
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 15/22
Fixed effects model in matrix notation II
Now stack the equations yi = ιαi + Xi γ + εi for the m time
series.
Next, define the mn × 1 vector y consisting of the stacked yi s,
define ε accordingly, define the mn × (k − 1) matrix X as the
matrix of stacked Xi and define the stacked mn × m matrix D as


ι 0 ··· 0


 0 ι ··· 0 
D=
.. 
 .. ..

. 
 . .
0 0 ···
ι
If α = (α1 , · · · , αm )′ , then following single regression model
arises
y = Xγ + Dα + ε, ε ∼ N (0, σ 2 I)
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 16/22
Fast Fixed Effects estimation in regression form I
Efficient estimators of α and γ can be obtained by OLS. When m
is large, direct OLS is computationally unattractive as it requires
the inverse of (X D)′ (X D) . An intuitive and easier method
(m+k−1)×(m+k−1)
applies partial regression, following the Frisch-Waugh theorem
(§3.2.5), in matrix notation:
• 1. ’Regress’ y and (all columns of) X on D and save the
residuals, MD y and MD X , MD = I − D(D′ D)−1 D′ . Since
(D′ D)−1 = n1 I , MD y and MD X have elements yit − y¯i and
x′it − x̄i ′ : just removing individual sample means!
• 2. Regress MD y on MD X to obtain γ̂OLS
γ̂OLS = (X ′ MD X)−1 X ′ MD y
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 17/22
Interpretation Fixed effects estimation in regression II
The Fixed Effect estimator or Least Squares Dummy Variable
Estimator (LSDVE) of γ is therefore obtained by regressing
unit-mean adjusted y on unit-mean adjusted X . The fixed effect
OLS estimates α̂ follow from the last m OLS normal equations
(3.41). In matrix regression form:
D′ X γ̂ + D′ Dα̂ = D′ y,
so that
α̂ = (D′ D)−1 (D′ y − D′ X γ̂).
which has the familiar interpretation of the estimates of constant
terms in regressions per individual (but here with a given
common γ ) :
α̂i = ȳi − x̄′i γ̂
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 18/22
Panel data model with random effects
The model with fixed effects cannot be consistently estimated if
n is fixed and m → ∞, and it cannot be used to forecast a new
unit ym+1 given xm+1 : αi is not modelled.
The simplest model for this purpose is the random effects
model, which has a random intercept with a common mean α
for all units. In social sciences (SPSS) this specification is called
called mixed model (mix of random and fixed coefficients).
αi = α + ηi ,
ηi ∼ IID(0, σα2 )
yit = α + x′it γ + ωit ωit = εit + ηi , εit ∼ IID(0, σ 2 )
with ηi and εit independent. The disturbances ωit are correlated
with their own past because of the ηi . The properties of ωit are:
2 ] = σ2 + σ2 ,
E[ωit ωis ] = σα2 for t 6= s
E[ωit ] = 0, E[ωit
α
E[ωit ωjs ] = 0 for all t, s and i 6= j
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 19/22
Random effects models FGLS, I
We can estimate the parameters α and γ by OLS, but this
estimator is not BLUE since the disturbances ωit are cross
correlated. An efficient estimator can be obained by feasible
GLS.
In the first step of FGLS we need to estimate σ 2 and σα2 .
Since ηi is fixed in the ith unit, it can be removed from the model
by taking the unit de-meaned variables. Consider
yit − ȳi = (xit − x̄i )′ γ + (εit − ε̄i ),
i = 1, . . . , m,
t = 1, . . . , n
Let γ̂ be the OLS estimate of γ for the above model. Then the
within variance, σ 2 = E(ε2it ), is estimated by
σ̂ 2 =
1
m(n − 1)
m X
n
X
(yit − ȳi − (xit − x̄i )′ γ̂)2 .
i=1 t=1
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 20/22
Random effects models FGLS, II
To estimate σα2 we combine the within variance estimator σ̂ 2
and the between variance estimator which estimates the
unexplained variance between unit-means in:
ȳi = α + x̄′i γ + (ε̄i + ηi ),
i = 1, . . . , m
2,
The variance estimate of this regression, denoted by σ̂B
2 and σ 2
estimates var(ε̄i + ηi ) = var(n−1 σ 2 + σα2 ). Combining σ̂B
one derives the estimator
2
σ̂α2 = σ̂B
− n−1 σ̂ 2
Given σ̂α2 and σ̂ 2 one can do the second step of FGLS to
reestimate α and γ . The resulting estimator is also known as the
EGLS (Estimated GLS) estimator of γ .
Exercise (3): Check the derivation on page 695-696 for
m = 3, n = 2.
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 21/22
Conclusion
The courses Econometrics I and II have introduced you to
• the main ideas of parametric econometric modelling
(Data analysis, parsimonious specification, consequences
of modellling errors, diagnostic checking, testing) and
• the basics of econometric (asymptotic) inference (Exact
statistical inference, likelihood based inference, moment
based inference, stationarity, rate of convergence)
in
• Static linear and nonlinear single equation models
• Binary Discrete choice models
• Dynamic linear single- and multiple equation models
• Panel data models
Many different parametric models and methods exist, but these
are (all) based on (combinations of) the ideas mentioned above.
Not discussed: Bayesian and nonparametric estimation and infererence
TI Econometrics II 2006/2007,
§7.7.1-7.7.3 – p. 22/22