CALDER Conference
Estimating Teacher Quality with
Administrative Data
Eric Hanushek
Texas Schools Project
University of Texas at Dallas
State of Knowledge on Teacher Quality

Little consistent impact of measured
characteristics





Teacher education
Teacher experience (past initial years)
Salary
Certification
Teacher test scores
Outcome Based Measures of Teacher
Quality

Move to outcome based measures of quality
Agnostic on characteristics
Consistency of “value-added”


c
iGs
A
   X
c
iG
 j   j 
c
iGs
 S
c
iGs
 j  e
c
iGs
State Administrative Data

Longitudinal information on individual student
growth
G 1
G 1
G 1
G 1
g 1
g 1
g 1
g 1
c
c
c
iGc    X igs
   Sigs
  igs
 ( ic   G  g  ic )


Value of large “n” and large “t”
Importance of institutional detail
Basic Results

Common finding: large variations in quality

Small number of years of good teacher have
enormous impacts
Figure 3. Kernal Density Estimates of Teacher Quality Distribution: Standardized Average Gains Compared to
Other Teachers at the Same Campus by Teacher Move Status
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
-2
-1.8
-1.6
-1.4
-1.2
-1
-0.8
Stays at Campus
-0.6
-0.4
-0.2
Campus Change
0
0.2
0.4
0.6
District Change
0.8
1
1.2
1.4
Out of Public Education
1.6
1.8
Basic Results

Common finding: large variations in quality




Small number of years of good teacher have
enormous impacts
Teacher entry path not so important
Math more influenced by schools than
reading
Some evidence that quality is observable
Issues in Estimation -- Selectivity



Students select schools
Teachers select schools
Principals assign teachers to classrooms
Approaches to Selection

Achievement growth models




Heterogeneity differences out
Add student fixed effects
Within district analyses
Within school analyses
Measurement Error

Tests are noisy


Normal reliability
Correlated errors



Common disturbances (“dog barking”)
Cheating
Food and nutrition
Tests are Imperfect



Not uniform across skill distribution
Accountability introduces biases
Multiple objectives
Importance and Analytical Approaches

Selectivity

Rivkin, Hanushek, Kain, Econometrica (2006)


Estimation across grades within schools
Hanushek, Kain, Rivkin, O’Brien, NBER (2005)

Estimation across classrooms within districts and
schools

0.11<σt<0.15
Measurement Error and Calculation of
Variance of Teacher Quality

Observe teachers in two years:


(1)
j ,

(2)
j
Correlation across years:
E (r12 )  var( ) / var( )
Estimated Variance in Teacher Quality
Within district
Within school and year
unadjusted
demographic
controls
Unadjusted
demographic
controls
Teacher-year variation
0.210
0.179
0.109
0.104
Adjacent year correlation
0.500
0.419
0.458
0.442
0.105
(0.32)
0.075
(0.27)
0.050
(0.22)
0.047
(0.22)
Teacher quality
variance / (s.d.)
Analytical approaches

Impact of between school selectivity


Can bound this within individual studies
Across state analyses

Different institutional structures



Reliability



Rules of school assignment
District structure
Sample sizes (by teacher/school/district)
External information
Teacher selection

District/school rules
Analytical approaches II

Alternative tests across states


Distinguish between testing v. other institutions
Change in tests by state
Conclusions




Research has produced substantial and
consistent findings
Value of longer “t”
Natural evolution to open new set of issues
Importance of institutional detail
TSP References

Rivkin, Steven G., Eric A. Hanushek, and John F. Kain.
2005. "Teachers, schools, and academic achievement."
Econometrica 73,no.2 (March):417-458.

Hanushek, Eric A., John F. Kain, Daniel M. O'Brien, and
Steve G. Rivkin. 2005. "The market for teacher quality."
Working Paper No. 11154, National Bureau of Economic
Research (February).
Figure 1. Relative Frequencies and Achievement Gains
by Score Interval of Initial Test Scores
25
1.00
0.80
Average Raw Gain
Frequency
Relative Frequency (%)
0.60
15
0.40
0.20
10
0.00
5
-0.20
0
-0.40
0-10
10-20
20-30
30-40
40-50
50-60
Initial Test Score Interval
60-70
70-80
80-90
90-100
Average Raw Gain (s.d.)
20