Can High Achievement be
Attributed to Better Teaching?
- Results of the TIMSS Video Study
Bogota, Columbia, November 2006
Frederick K.S. Leung
The University of Hong Kong
Introduction
East Asian students have consistently out-performed
their counterparts around the world in international
comparisons of mathematics achievement.
 Can the high achievement be explained by better
teaching in the East Asian classroom?
 This presentation reports some of the results of the
TIMSS Video Study in an attempt to portray the
mathematics teaching in the East Asian classroom
 Implications of the findings will also be discussed

TIMSS 1999 Video Study (Math)
Goal:
Describe and compare eighth-grade mathematics
teaching across seven countries (Australia,
Czech Republic, Hong Kong SAR, Japan*,
Netherlands, Switzerland, United States)
* The 1995 Japanese data were re-analyzed using the
1999 methodology in some of the analysis
Sampling and Data Collection
National probability sample of 8th-grade math
lessons: a Video Survey
 One lesson per teacher
 Sampled across the school year
 Standardized camera procedures
 638 lessons, from 50 (Japan) – 140
(Switzerland)

Data Coding and Analysis
An international team developed codes to apply
to the video data.
 Fluently bilingual coders in the international
video coding team applied 45 codes in seven
coding passes to each of the videotaped lessons.
 Three marks (i.e., the in-point, out-point, and
category) were evaluated and included in the
measures of reliability.
 If, after numerous attempts, reliability measures
fell below the minimum acceptable standard, the
code was dropped from the study.

The Mathematics Quality Analysis Group
A specialist group in mathematics and teaching
mathematics at the post-secondary level reviewed a
randomly selected subset of 120 lessons (20 lessons
from each country except Japan).
 The international video coding team created
expanded lesson tables for each lesson in this subset.
 The tables included details about the classroom
interaction, the nature of the math problems worked
on, mathematical generalizations, and other relevant
information.
 The tables were “country-blind,” with all indicators
that might reveal the country removed.

Instructional Practices in East Asia as
Portrayed by the Analysis of the Codes
1. Dominance of teacher talk


In all countries in the study, the teachers did a
lot of talking, and considerably more than their
students
Hong Kong and Japan differ considerably in
the amount of teacher talk
Average Number of Teacher and Student
Words Per Lesson
7000
6000
5536
5452
5902
5798
5148
5360
5000
4000
3000
2000
1000
810
824
640
766
1016
1018
0
AU
CZ
HK
Average number of teacher words
JP
NL
US
Average number of student words
Ratio of teacher and student talk
Hong Kong and Japanese teachers spoke much
more relative to their students
 “Hong Kong SAR eighth-grade mathematics
teachers spoke significantly more words relative
to their students (16:1) than did teachers in
Australia (9:1), the Czech Republic (9:1), and
the United States (8:1).” (p. 109, Chapter 5)
 When we factor in the relatively large class size
(about 40), the reticence of East Asian students
is striking

Number of Teacher Words Per 1 Student Word
Average Number of Teacher Words to
Every One Student Word Per Lesson
20
16
16
13
12
9
9
AU
CZ
10
8
8
4
0
HK
JP
NL
US
2. More opportunities to learn new
content
75% of lesson time in the East Asian
classroom spent on dealing with new content
 Corresponding figures for other countries
ranged between 42% (Czech Republic) and
63% (Switzerland)
 Inference: East Asian students learn more
mathematics than students in other countries?

Average percentage of lesson time
devoted to various purposes
3. Mathematics problems worked on
more complex
Procedural complexity of problems: “the number
of steps it takes to solve a problem using a
common solution method” (p.70)
 Japanese students worked on procedurally more
complex problems
 Problems Hong Kong students worked on not
particularly complex, although the percentage
(63%) of low complexity problems is relatively
small

Average percentage of problems at each
level of procedural complexity
Percent of Problems
100
80
8
11
8
25
29
16
6
39
25
12
22
6
27
60
40
45
77
64
63
69
65
67
20
17
0
AU
CZ
HK
JP
NL
SW
US
Low C omple xity
Mode rate C omple xity
High C omple xity
Problem complexity (cont’d)
Another measure of problem complexity:
length of time students spent working on the
problem (more or less than 45 seconds)
 Conclusion: East Asian students have more
opportunities to work on procedurally more
complex problems which required a longer
duration to solve

Average percentage of problems that
were worked on longer more than 45 s
4. Problems unrelated to real-life
Majority of problems in the East Asian
classroom were expressed in mathematical
language and symbols, and set in contexts
unrelated to real life
 Similar to classrooms in Czech Republic, and
differ markedly from classrooms in the
Netherlands

Average Percentage of Problems Per Lesson
Set Up With a Real Life Connection or With
Mathematical Language or Symbols Only
5. More proof
Problems East Asian students worked on
involved more proof
 The emphasis is particularly marked in Japan
 The practice in Hong Kong more in line with
Switzerland

Percentage of problems that
contained at least one proof
Instructional practices as portrayed
by the analysis of the codes
• Dominance of teacher talk
• Students have more opportunities to learn
new content
• Students solve problems that are more
complex and are unrelated to real-life
• More proof
Quality of Content as judged by the
Math Quality Analysis Group
(based on the same data set)
Japanese not in the analysis
“Readers are urged to be cautious in their
interpretations of these results because the subsample, due to its relatively small size, might not
be representative of the entire sample or of
eighth-grade mathematics lessons in each
country.” (p. 190, Appendix D)

1. Relatively advanced content
“the ratings for countries with the most advanced
(5) to the most elementary (1) content in the subsample of lessons, were the Czech Republic and
Hong Kong SAR (3.7), Switzerland (3.0), the
Netherlands (2.9), the United States (2.7), and
Australia (2.5)” (p. 191, Appendix D)
Percent of Sub-sampled Lessons
Percentage of Lessons in Sub-sample at
each Content Level
100
15
0
0
20
20
80
5
0
20
30
35
30
60
35
45
40
40
40
30
45
20
20
0
10
15
AU
CZ
40
0
5 0
HK
25
20
15
10
15
15
NL
SW
US
Advanced
Moderate/Advanced
Moderate
Elementary/Moderate
Elementary
2. More deductive reasoning

Deduction reasoning = “deriving conclusions
from stated assumptions using a logical chain
of inferences.”

The reasoning did not need to include a formal
proof, only a logical chain of inferences with
some explanation.
Percent of Sub-sampled Lessons
Percentage of Lessons in Sub-sample
that Contained Deductive Reasoning
100
80
60
40
15
20
0
0
AU
5
CZ
10
10
SW
US
5
HK
NL
3. More coherent
Coherence was defined by the group as the
(implicit and explicit) interrelation of all
mathematical components of the lesson.
Percent of Sub-sampled Lessons
Percentage of Lessons in Sub-sample
Rated at Each Level of Coherence
100
30
80
30
55
60
60
20
40
15
20
15
0
AU
Mixed
5
CZ
15
Moderately fragmented
5
20
0
Moderately thematic
20
10
10
15
65
90
30
Thematic
0
10 0
0
HK
10
5
NL
20
Fragmented
35
10 0
SW
0
US
4. More fully developed presentation
Presentation = “the extent to which the lesson
included some development of the mathematical
concepts or procedures”.
 Development required that mathematical reasons or
justifications were given for the mathematical
results presented or used.
 Presentation ratings took into account the quality of
mathematical arguments.
 Higher ratings meant that sound mathematical
reasons were provided by the teacher (or students)
for concepts and procedures.
 Mathematical errors made by the teacher reduced
the ratings.

Percent of Sub-sampled Lessons
Percentage of Lessons in Sub-sample
Rated at Each Level of Presentation
100
80
0
40
5
10
20
15
20
30
45
55
30
0
AU
10
Substantially developed
20
Moderately developed
Partially developed
20
10
Fully developed
40
35
20
25
30
60
40
5
15
20
0
CZ
10 0
HK
20
5
10
15
NL
SW
40
US
Undeveloped
5. Students more likely to be engaged
Student engagement = “the likelihood that students
would be actively engaged in meaningful
mathematics during the lesson”.
 A rating of very unlikely (1) indicated a lesson in
which students were asked to work on few of the
problems and those problems did not appear to
stimulate reflection on math concepts or procedures.
 A rating of very likely (5) indicated a lesson in
which students were expected to work actively on,
and make progress solving, problems that appeared
to raise interesting mathematical questions for them
and then to discuss their solutions with the class.

Percent of Sub-sampled Lessons
Percentage of Lessons in Sub-sample Rated at
Each Level of Student Engagement
100
80
10
10
35
15
20
10
15
25
Probable
55
30
30
30
Possible
30
20
30
0
0
AU
10
5
CZ
30
5 0
HK
0
Very likely
20
40
60
40
10
45
30
Doubtful
Very unlikely
10
10
10
15
NL
SW
US
6. Overall quality
Overall quality judgment:
“the opportunities that the lesson provided for
students to construct important mathematical
understandings” (p. 199, Appendix D)
“the relative standing of Hong Kong SAR was
consistently high ….” (p. 200, Appendix D)
Percent of Sub-sampled Lessons
Percentage of Lessons in Sub-sample Rated at
Each Level of Overall Quality
100
80
5
30
30
15
20
0
25
High
35
40
60
20
35
30
40
20
5
15
45
20
15
15
15
5
0
AU
CZ
15
25
10 0
HK
15
40
10
NL
SW
Moderate
Moderately low
25
20
20
Moderately high
US
Low
General Ratings for Each Overall Dimension
of Content Quality of Lessons
5.0
4.0
3.0
HK
SW
AU
NL
CZ
US
HK
SW
CZ
AU
NL
US
HK
CZ
SW
AU
NL
US
2.0
HK
CZ
SW
AU
NL
US
1.0
0.0
Coherence
Presentation
Student
engagement
Overall quality
AU
CZ
HK
NL
SW
US
Quality of the Content as judged by
the Math Quality Analysis Group
 Relatively
advanced content
 More deductive reasoning
 More coherent
 More fully developed presentation
 Students are more engaged, and
 Overall quality is high
Discussion
Some characteristics of the East Asian classroom
found in this study (large class size, dominance of
teacher talk, reticence of students, abstract
problems unrelated to real-life) seem to be at odds
with modern theories of learning
 Despite the rhetoric of constructivism and studentcentred learning to the contrary, the findings show
that meaningful learning can still take place in a
teacher directed classroom with a large class size
 Teacher dominance with a lot of teacher talk does
not necessarily lead to passive, receptive learning

Much depends on the content of the teacher talk
and how it is delivered, and whether the talk can
stimulate students to be engaged in mathematics
 The data in this study suggest that the kind of
teacher talk in the East Asian classroom was able
to direct students to be engaged in the lesson
 Indeed, a well-taught teacher-dominated lesson
may better provide the mathematical coherence
which students need in their construction of
mathematical knowledge rather more effectively
than many student-led approaches.

Mathematics content covered in East
Asian classrooms

East Asian students learned more new content
than their counterparts in the West

The content was more complex and advanced

There were more proofs and more use of
mathematical language
Proof and the use of maths language
In many countries, mathematical language is
considered too alien and proof too abstract for
school students
 Both are deemed to be too difficult for school
students and are thus excluded from the curricula
 However, both have traditionally been regarded as
distinctive features of mathematics, and it seems that
they are still judged to be so in the East Asian
classroom
 Neither was stressed in TIMSS and PISA


A firm foundation in mathematics laid for East
Asian students through emphasis on mathematical
language and proof that enables these students to
do well in the less abstract tasks in the
international tests?
“In a milieu which seems to believe that the most effective
way to enhance understanding and raise attainment levels
is through an improved pedagogy, the clear indication that
the high achievement of East Asian students is related to
the high quality of the mathematics content to which they
are exposed, should act as a sharp reminder that without
quality content, quality learning will not take place - no
matter how ingenious the teaching method.”
Expectation on students
East Asian teachers have higher expectations of their
students on the kind of mathematics to be learned
 The level of expected mathematics achievement in
many Western countries seems to be declining
 Mathematics is considered by students and teachers
alike as a difficult subject
 Majority of student population not expected to learn
more advanced mathematics, and are not even
expected to do well in elementary mathematics
 The low student achievement becomes a selffulfilling prophecy

Teacher competence
East Asia teachers are sufficiently competent in
mathematics to deliver complex and advanced
content (Ma, 1999, Leung and Park, 2002)?
 More coherent and better developed presentation
may be attributed to the mathematical and
pedagogical competence of the teachers
 Ma (1999): competence in mathematics and
pedagogy are intrinsically related: without a
profound understanding of mathematics, it is not
possible to invoke the appropriate pedagogy.

Scholar teacher
In East Asian or “Confucian Heritage” Culture
(Biggs, 1996), the ideal of the “scholar teacher”
is that of an expert or a learned figure in the
subject matter
 Teaching skills are also important, but teachers
will not be respected if they are not expert in the
area they teach
 This image of the scholar-teacher may provide
incentives for East Asian teachers to strive to
attain high levels of competence in the subject
matter as well as in pedagogy

Conclusion
No simple casual relation between classroom
teaching and student achievement can be drawn,
but East Asian teachers did teach differently from
their counterparts in the West
 Classroom practices are deeply rooted in the
underlying cultural values of the classroom and the
wider society
 Simple transplant of educational practice from
high achieving countries to low achieving ones
would not work
 One cannot transplant the practice without
transplanting the culture as well

Conclusion (cont’d)
We should identify not only the superficial
differences in educational practice, but the intricate
relationship between the educational practice and
the underlying culture of other countries
 Through identifying the commonality and
differences of both the educational practices and
the underlying cultures, we may then determine
how much can or cannot be borrowed from another
culture.

My e-mail address:
[email protected]