The impact of using graphic organizers
to test reading comprehension
Marci Nelson Özer
Bilkent University School of English Language
(BUSEL)
Ankara, Turkey
EALTA conference
May 24, 2013
The impact of using graphic
organizers to test reading
comprehension
 Why graphic organizers?
 Graphic organizers at BUSEL
 The study
 Implications
 Further research
Reading Comprehension
Construct: Reading
is the skill of
comprehending and
interpreting directly
stated and/or
inferred information
from a written text.
This involves
employing reading
strategies as well as
grammatical, lexical
and discoursal
knowledge.
BUSEL syllabus
Koda (2005)
Graphic Organizers
Classification
Group 1
Cause & Effect
Group 2
Flow Chart
Hierarchy
Why graphic organizers?





Focus is on interrelationships of ideas rather than
isolated ideas (Kinchin, Hay, & Adams, 2000)
Graphic organizers get away from the linearity of the
text and approximate the macrostructure of the text
(Chang, Sung, & Chen, 2002)
Texts are not listlike sequences—they have rhetorical
structures that organize information and serve writers’
purposes (Grabe, 2009)
They allow a holistic understanding that words alone
cannot convey (Jones, Pierce, & Hunter, 1988-1989 –
in Jiang & Grabe 2007)
They have powerful construct validity (Grabe, 2009)
4-option multiple choice
Graphic organizer
Graphic organizers at BUSEL
Class work: organizing notes from reading,
organizing ideas before writing, awareness of text
structure and idea structure
• At the B1 level
Reading for information and argument: can
recognize significant points in straightforward
newspaper articles on familiar subjects
 Test: open-ended cause and effect table

Teacher feedback
Positive
Negative
 The students needed to

Some students missed
everything in Reading 2
(graphic organizer), although
they did quite well on Reading
1. I’m afraid the task surprised
them and affected them
negatively.

Some students couldn’t do the
task fully. There were more
blank answers than wrong
answers.

Some students were not able to
match the reasons with their
effects because they couldn’t
figure out the function of the
arrows on the chart.
analyze the text, so they
needed to comprehend the
text
 It shows students that they
need to find other ways to
understand the reading than
just finding answers
 Most students regard
reading as an ability to
answer a set of open-ended
and multiple choice
questions. With the new
task type, these beliefs can
begin to change…
Group A
201 students
THE STUDY
Group B
203 students
Reading 1
Graphic
organizer
(cause & effect)
Multiple choice
questions
Reading 2
Graphic
organizer
(hierarchy)
Multiple choice
questions
Completed without the texts:
•T/F/NM summary of Reading 1
•Multiple choice summary of Reading 2
Multiple choice questions
Excerpt from Reading 2:
The Sumerians were also the
first people to build vehicles with
wooden wheels. The invention of the
wheel greatly improved their daily lives.
Before the invention of the wheel,
people had to pull their goods and it
was very difficult to do this. However,
after they built vehicles with wooden
wheels and attached them to horses or
donkeys, they could transport goods
more easily over long distances. The
wheel also increased the speed and
strength of Sumerian armies. During
battles, drivers could stand on the top
of the vehicle while driving and throw
spears at enemies.
Sample questions:
What does “this” in paragraph 4 refer
to?
A
B
C
D
pulling their goods
inventing the wheel.
daily life
building vehicles
The wheel benefitted Sumerians by
helping them…
A
B
C
D
treat their animals better.
develop their fighting techniques.
improve their relationships
hide from their enemies.
Graphic organizer questions
Excerpt from Reading 2:
The Sumerians were also the
first people to build vehicles with
wooden wheels. The invention of the
wheel greatly improved their daily
lives. Before the invention of the
wheel, people had to pull their goods
and it was very difficult to do this.
However, after they built vehicles with
wooden wheels and attached them to
horses or donkeys, they could
transport goods more easily over long
distances. The wheel also increased
the speed and strength of Sumerian
armies. During battles, drivers could
stand on the top of the vehicle while
driving and throw spears at enemies.
Summary Task
SUMERIAN CULTURE
Complete the summary based on what
you remember from the text.
The Sumerians were an ancient
culture that lived in Mesopotamia from 3500
to 1750 BC. They made many discoveries
because of their _______(27).
They
invented the first system of writing, which
helped them to develop _______(28) and
_______(29). The Sumerians also invented
_______(30), which helped them fight better
against their enemies. They also developed
a mathematical system based on the number
_______(31). This helped them become
better at _______(32). Today, Sumerians
are known for their _______(33).
27
A) central location
B) difficult environment
C) great leaders
28
A) poetry and songs
B) relationships
C) paper
29
A) laws
B) education
C) health practices
30
A) weapons
B) the military
C) the wheel
31
A) 80
B) 60
C) 10
32
A) farming
B) travelling
C) learning
33
A) interesting culture
B) spoken language
C) useful developments
Results
READING
1
Questio
n type
Facility Point
Value
Biseria
Mean
l Mean
Summary Facility
Task Value
Mean
Group A
M/C
77.3
21.2
67.7
23.8
Group B
G/O
72.1
47.2
67.7
26.1
READING
2
Questio
n type
Facility Point
Value
Biseria
Mean
l Mean
Group A
G/O
70.2
41.4
85.9
32
Group B
M/C
70.8
33.1
85.7
21.3
Summary Facility
Task Value
Mean
Point
Biserial
Mean
Point
Biserial
Mean
T-test – Reading 1 and Reading 2
Group A
N
Mean
Std.
Deviation
Std. Err
Mean
Reading 1
MC
201
4.6318
1.09260
.07707
Reading 2
GO
201
4.2119
1.30367
.09195
N
Mean
Std.
Deviation
Std. Err
Mean
Group B
Reading 1
GO
203
4.7438
1.59622
.11203
Reading 2
MC
203
4.0764
1.06037
.07442
T-test – Summary 1 and Summary 2
Group A
N
Mean
Std.
Deviation
Std. Err
Mean
Summary
R1
MC
201
4.7463
1.26107
.08895
Summary
R2
GO
201
6.0299
1.15287
.08132
N
Mean
Std.
Deviation
Std. Err
Mean
Group B
Summary
R1
GO
203
4.7438
1.22410
.08591
Summary
R2
MC
203
5.9951
.98251
.06896
Tentative conclusion
 Graphic organizers as a reading exam task seem
to discriminate better between weak and strong
readers
For further consideration
Graphic organizers…
 can help raise awareness of discourse structure
 may be a good way to test reading fluency
 seem to require readers to confront the text for
information rather than relying on a series of
questions to guide them
Issues & Further Research
Cognitive overload?
 Training for teachers and students
 Experimenting with format to ensure validity and
reliability
 How to continue and develop GO use at higher levels
(B1+ / B2)

Future studies will involve…
Student interviews
 Think aloud protocols
 Teachers’ perceptions

THANK YOU
[email protected]
References
 Chang, K., Sung, Y., and Chen, I. (2002) “ The effect of concept mapping to
enhance text comprehension and summarization,” The Journal of Experimental
Education, 71(1), 5-23
 Grabe, W. (2009) Reading in a Second Language, Cambridge: Cambridge
University Press
 Jiang, X. and Grabe, W. (2007) “Graphic organizers in reading instruction:
Reading findings and issues” Reading in a Foreign Language, Volume19,
Number 1, April 2007
 Kinchin, I., Hay, D., and Adams, A. (2000) How a qualitatative approach to
concept map analysis can be used to aid learning by illustrating patterns of
conceptual development” Educational Research, Vol. 42 No. 1, Spring 2000 4357
 Koda, K. (2005) Insights into Second Language Reading, Cambridge: Cambridge
University Press