San Fernando Valley State College
A Study oL the Relationship
Between the Degree of Difficulty and a
Student's Performance
Using the Gloze Method
Ol
Testing
Readability
in
Grades 5, 6, 7, and 8
A
project submitted in partial IUJ.fillment of the
requirements for the degree of Master of .t<.ducation
in Reading Improvement
by
Howard P. Solberg
January, 1971
TABLE OF CONTENTS
PAGE
LIST OF TABLES ....................... •- ••• e- . . . . . . . . . . . ,. •
•.
iii
THE PROBLEM•••••••••••••••••••••••••••••••••••
1
CHAPTER
1.
PURPOSE OF STUDY. • ~- .. , a .. •- ••
• •••• e a • • a • • ,
2
• • • ., • • • •- • • • • • •
3
DEFINITIONS OF TERMS USED••••••••••••••
3
LIMITATIONS OF THE STUDY•••••••••••••••
5
2.
REVIEW OF THE LITERATURE••••••••••••••••••••••
6
3.
METHOD AND MATERIAL ••••••• • • • • •- •- ,_ .......... • •••
13
NEEDED ..... •· ••• • • •-. •- .......... • •• , • • •
13
SOURCE OF DATA\ ••• • •• •- ••• "•. • ..... • •• • .. •
16
TOOLS AND TECHNIQUES•••••••••••••••••••
16
DATA•••••••••••••o•••••••••~>ooooeo
21
HYPOTHESIS • •- • •-. •- •• •- •• •- a- ._ •
DATA
4.
ANALYSIS OF
s.
DISCUSS ION ••••••••• , •••••••••••
6.
e ••••••••••••• •
34
CONCLUSIONS • , •• • , •••• , •• • ••••• •-. • • .. • • • •
36
IMPLICATIONS FOR FURTHER STUDY•••••a•••
37
SUMMARY • .... , ••• •·• •• • ................... •. • •••
e.
38
BIBLIOGRAPHY •. •-• ••• •·•. a ••• •·• .... " ••••••• • • o • o • o
40
APPEND IX A ...................... , •- a
•- . . . . . . . . . • • e •- •
43
CLOZE PASSAGES •••.• • •••• •· •. • • • • • • • • • • • • • •
43
APPEND IX JJ'.............. • •- •- • , ••••• •- ••••••• •- • •- ••• •
51
TABULATION OF ORIGINAL DATA •••• • • ., •. • •.
51
ii
Ll ST OF TABLES
Page
TABLE
Readabl I tty levels as predicted by the Flesch
Reading EAse Formula, the Fry Readabil tty Graph
and the DA I e Cha I I Rea dab I II ty Formu Ia Showing
the Average readability level of each passage ••••••••• 6a
II
nt7yh~
Readab,U·t ty Chart ••••••••••••••••••••••••••••••• I0
Ill
Dale-Chall Readability Formula ••• •••••• ••••••••••••••• 12
IV
Flesch Formula ...................................... •••• 14
v
Flesch Reading Ease Scale .............................. l5
VI
Gloze Test Uirectlons ••••••••••••••••••••••••••••••••• IU
VII
Grade level of passa8es according to publishers and
rea dab I I i ty formu Ia·s •••••••••••••••••••••••••••••••••• 21
VIII
Average Scores on cloze tests ••••••••••••••••••••••••• 22
IX
Composite Scores on Passage 0 ••••••••••••••••••••••••• 23
X
Composite Scores on Pass.age E.......................... 24
XI
Composite of Scores on Passage F•••••••••••••••••••••• 25
XII
Composite Scores on Passage G••••••••••••••••••••••••• 26
XIII
Composite Score on Passage H•••••••••••••••••••••••••• 27
XIV
Average class scores on the
California Testof Mental Maturity ••••••••••••••••••••• 29
XV
Equlvaient cloze and multiple-choice test
percentage scores ••••••••••••••••••••••••••••••••••••. 30
XVI
Class scores compared to
~ormutn's
equivalent scores •••••••••••••••••••••••••••••••••••• ,31
iiI
Chapter I
THE PROBLEM
The schools within tne Southern California area have been
attacked during the past few years on the basis that the children
in these schoo Is are not Iearn i.ng as we II as they cou Id.
This
criticism has gone so far as to reach tne state's legislative
powers as well as the_ governor or tne state.
In the past election,
the State Superintendent of rub I ic Instruction, Max Rafferty, lost
to Wilson Riles on the basis of tailing to Instruct our students.
Many teachers, in the course of their conversation have
expressed their thoughts that the state textoooks
aFe~not
written
in a manner so that their classroom students can comprehend their
meaning.
While this may sound as though it were a local problem,
as one reads about education
thr~ughout
the nation, it seems as if
practically all or the scnool systems are on the defensives.
The
idea of the schools throughout the nation being attacked for not
dorng a good job of teachi.ng is very disturbing.
Perhaps the problems within out schools today are not entirely the fault of the teachers involved but also the fault of the
materials that, by law, they must work with.
logical conclusion for a teacher to arrive at.
be wrong with the material?
This seems like a
But just what could
Is it written at too high a level; is it
written at too low a level; does It demand too much from the children
or not
en~ugh?
Is there a point where material becomes too hard for
the student causing frustration, as wei I as tao easy for him
causing him to feel bored and belittled.
The problem that educators face is try lng to determine the
answers to the above.di lemmas.
PURPOSE OF STUDY
John Bormuth's proposal to the U. S. Commissioner of Education
entitled "Reference Criterion of Readabi I ity"
of finding at what
I
deals with the problem
reading level a student simultaneously gains a
maximum amount of new information from the material whl le at the
same time exhibits a maximum of Interest in using these materials In
his studies.
Later on in his dissertation he quotes a related study by
2
Coleman (1968)
in which Coleman states an apparent tendency for
subjects to exhibit tess Information gain on very easy passages than
on the passages In the Intermediate range of difficulty.
In regard
to this statement, Bormuth states "if Coleman's drop in Information
gain on his easy passages was real, It may mean that the shape of
the gain-to-difficulty relationship changes as a result of an interaction between passage difficulty and reading ability.
It seems
strongly advisable that further studies vary both the reading abl lit~s
of the subjects and the difficulties of the passages In such
1
Bormuth, John, "Reference Criterion of Readab i I it 'tProposal for Research Submitted to the U. s. Commissioner of Education for Support through Author! zat ion of the Bureau of Research,"
The University of Chicago, Chicago, I I linois, September 30,1970.
2
~.,
p. 6.
3
a manner that this interaction wil I be demonstrated, it in fact It
does exist.
3
In a study by Bloomer made from specially written multiple
choice tests given after 24 cloze exercises, he concluded that tests
made from very easy materials were less "motivating" than materials
more closely associated with the grade level of the subjects.
Coleman states that coll_ege sophomores received maximum information gain when reading passages written at approximately a fifth
4
grade level of dlfficulty.
This, then is t.he purpose of the study- - to try to determine
whether elementary school children gain more when the work is at
their own reading level rather than when the work is below their
reading grade level.
HYPOTHESIS
A student wil I show evidence of better grades working with
a lower level of reading material than he will by using reading
material at his own grade level or above.
DEFINITION OF TERMS USED
~~ ~·
word.
The blank which is substituted for a deleted
Blanks in a test are always of the same length.
3
Thomas Potter, "A Taxonomy of Gloze Research, Part I:
Rea dab i I i ty and Reading Camp rehens ion", Southwest Region a I Laboratory for Educational Research and Development, Inglewood, Cal. 1968.
P• 17.
4
coleman, E., and Miller, G., "A Measure of Information Gained
During Prose Learning," Reading Research Quarter Iy 3; 369-386. 1968.
4
Gloze Procedure.
The set of rules by which cloze tests
are made.
Gloze Readabl llty.
The number of correct responses on a
cloze test over that passage.
Gloze Response.
What the student writes or fal Is to write
in a cloze Item.
Gloze
~·
A col lectlon of cloze items.
fifty cloze items on a cloze test.
There are usually
5
Intermediate Grades. The educational level 06 groups which
include the fifth, sixth, seventh, and eighth grades.
Readabl llty.
According to Chat 17 readabl I tty is the sum to-
tal of alI the elements within a given piece of printed material that
affect the success a group of readers have with it.
The success the
readers have is n1e extent to whIch they understand It, read it at a
maximum speed and find It interesting.
According to Holt's dlcttonary, 8
the material must be I) easy to read, 2) Interesting or pleasant, and
3) legible and or plainly written.
Readabl I lty Formulas.
Method of measurement intended as a
predictive device, the design of which Is Intended to provide quantitative, obJective estimates of difficulty for pieces of writing without requiring the students to read It or take tests on lt.
5
John Bormuth, "Gloze Tests as Measures of Readabl lity
and Comprehension Ability," Indiana University, Aug. 1962. p. 3.
6
George K Iare, The Measurement .£.!. Rea dab i I i ty_, Ames, Iow a,
1963. pp. 33-34.
7oa le and Cha II, .,~1fhe l &ontCl:ep:t~:of Readab i I ity, 11 Elementary
Engl is~, 25:23, January, T9'49:-'
8.Ib2. Holt Intermediate Dictionary£.!_ Ameri.~ Eng I isb'~· (New
York: Holt, Rinehart and Winston, Inc. 1967), p. 657.
5
LIMITATIONS OF THE STUDY
This study was conducted in a smal I school district located
In a mountain community north of the urban center of the Los Angeles,
California, area.
The area could be classified as rural and/or
recreational resort property.
the lower middle class.
A sizeable proportion of the students tested
(8%> are from foster homes.
In this area.
vision at home.
The population would tend to fa I I In
Television reception is extremely poor
Therefore, only about 25% of the community has teleDue to the above conditions, there Is a doubt as
to whether or not the results of this study could be a measurement of
nationwide performance.
Other limiting factors would be the manner In which the
directions for the test were given, the Importance placed upon the
test by the teacher, etc.
Readabl llty formulas themselves are not exact In their
measurement of reading levels, although as shown In Table I, the
results do tend to complement each other.
Chapter 2
REVIEW OF THE LITERATURE
In reviewing related I iterature In the area of readability
it was found that whl le there are many formulas that measure readabi I tty, the three most widely used, (both for rellabl I tty as well as
ease of administration) were the Dale-Chal I Formula, the Fry Readabi I tty Chart, and the Flesch Reading Ease Formula.
The Flesch Formula became widely known as the author wrote
many books on rea dab I II ty.
He fe It ext st i.ng formu Ias haa many weak-
nesses In them In that they were not suitable for adult material,
there was too much emphasis placed on vocabulary at the expense of
otho::~r
ractors and too little attention was placed on abstract words.-
The results of hls studies were published in his
book,~ To~
Rea dab i II ty. 9
In IY50, Flesch considered a new factor of readaoi lity, abstraction.
While this was Impractical for common usage (he devised
a formula to determine the relationship of definite words and the
level of abstractlonl, It was found helpful in certain situations.
In IY57 and 1958, Flesch devised two new formulas designed
mainly for writers.
readabl lit
In 1943, Flesch developed two regression
formulas whl le at
~olumbla
University.
Formula A was
a measuring device for predicting reading ease and Formula B was a
9
Rudolph Flesch,
Brothers, 1951.
~.!2,~
Readability, New york; Harper and
TABLE I
Readability Levels as Predicted by the
Flesch Reading Ease Formula, the Fry
Readability Graph, and the Dale Chall
Readability Formula Showing the Average
Readability Level of Each Passage
H
8.3
G
00
w
~
~
F
~
<
00
~
G
E
<
00
00
<
~
D
3
4
5
GRADE
6 a
6
7
L EV E L
8
9
7
device for measuring human Interest.
were based upon three variables:
The two regression formulas
IJ
sentence length in words,
2) number of affixes, and 3) number of references to people.
Both
formulas were revised In 1951 due to shortcomings in the structure
of the original formulas and difficulties in applying them.
In the
reading ease formula, the affix count per 100 words was found to have
a correlation of .87 with the syllable count per lOu words, so he
substituted the syl table count for the count of affixes.
The
syl table count was also considered as a measure of abstraction, as
the correlation between affixes and that of abstract words was found
to be .18.
The average sentence length in words, as used in the
original formula was retained as a measurement of sentence complexity.
The final formula Is:
RE
= 206.835
- .1346 WI
-
1.015 sl
RE = Pred l cted Reading Ease
wl
= .number of sy I Iab Ies p e r hundred words
sl
= average sentence length per hundred words
The score computed by thls formula may have a range from zero to one
hundred.
A.score of 100 on the reading ease formula indicates that
a person who has completed the fourth grade could be predicted as
being able to answer correctly three-fourths of the test questions to
be asked about the passage rated, whl lea score of fifty words Indicate
that a person having completed the ninth through eleventh grades would
be able to answer correctly three-fourths
asked about the passage rated.
to one-tenth of a. grade.
holds true untl I
abou~the
m the
test questions to be
One point on the scale would correspond
According to Flesch, this relationship
seventh grade, at which ttme the formula
8
under-rates the grade level.
Studies done by Klare
10
and Margaret Peterson 11 indicate
that the Flesch Formula ts a valid measurement as compared to
other Readabl llty Formulas.
At best, however, any formula provides
only a rough estimate of readabl llty.
In 1948, Edgar Dale and Jeanne Chal I designed a formula
which was intended to correct the deficiencies of the original
Flesch Formula.
They postulated that a larger word list would pre-
diet better than the original Dale ltst of 769 words or counts of
affixes, personal references would be unnecessary and that a more
efficient formula could be developed using the word and sentence
factors.
This formula they devised was Intended prlmari ly for
adult materials.
They found, however, after experimenttng with
the formula, that It gave a grade level range from third grade to
twelfth.
12
Their formula is:
x
=
.1579x
1
+ .0496x + 3.6365
2
X= Reading score of pupil
X =% of words not on Dale (3000) word list
1
X2=
Aver:age sentence length in words
Evidence of validity was offered by the authors by showing
evidence of high correlation of judgments of teachers, reading
10
KI are , £E. ill. , p • I I 8 •
11
Margaret Peterson, "Comparison of Flesch Readability Scores
with a Test of Reading Comprehension," Journal £!.Applied Psychology
4: 35 - 36, Feb. 1956.
12
Jeanne Chall, Readability: An Appraisal of Research and
Application. Columbus: Ohio University, 1958, pp. 31-34.
9
specialists, and reading grades of children who were able to ansv1er
13
correctly 75% of the questions on the passages selected.
Many
studies of comparative validity have been done since.
In terms of
grade placement, the findings show the Dale-Chall and Flesch Reading
Ease formula: to be the most consistent.
Edward Fry from Rutgers University;, has
mula which ts expressed in the form of a graph.
also devised a for(see Table I 1).
The two factors used are Vocabulary Load and Grammatical Complexity.
The number of words In sentences and the number of ;syl tables are
placed on a graph to determine the readability level.
were determined by plotting clusters of books.
Grade levels
Through the graph
is only an estimate, Fry feels that Its \simplicity makes it very
usable.
Comparative validity studies by Kistulentz In 1967, showed
correlations ranging from .78 to .96 with student comprehension of
tenth graders.
Another comparative val !dation has been done by Fry
on primary reading material using the cloze procedure, the Spache
Formula and oral reading tests.
Correlation ranged from .90 to .96.
The Gloze procedure was developed by Wilson Taylor in 1953.
The word "cloze" was taken from the Gestault psychologist's concept
of
clozure which Is to fi II In the parts of an incomplete sentence
in order to arrive at the correct solution.
Taylor reasoned that if
a student could understand enough of the meaning of a printed
passage when words were removed so that he could replace the missing
13
1bld
_,
p. 91.
14
Edward Fry, !:_ Rea dab I lity Formu Ia ~ Saves Time, Journa I
of Reading, II :513-17, 575-78, flpri I, 1958.
14
FRY'S READABILITY CHART EXTENDED THRU PREPRIMER LEVEL
Number of s~l1ables
100 words
0
0
~
Ol
"0
~
0
:;:.
0
0
..-l
-~
-(()
LUP.,
_J
cnw
<:t:<Ll
1-0
s::(()
.J-1
~
(()
til
\1-1
0
S4
(()
.0
a
a
Q)
!'J()
tlj
£.1
(()
.I>
<
-::t
0
..-l
00
0
..-l
N
~
..-l
0
C'l
...::!"
C'l
00
N
..-l
N
\0
C'l')
C'l')
0
...::!"
...::!"
...::!"
00
...::!"
N
1.()
\0
0
1.()
\0
-::t
N
.....
00
\0
\0
\0
.....
40.0
35.0
30.0
23.0
20.0
16.7
14.3
12.5
11.1
10.0
9.2
0
8.3
7.8
7.1
6.7
6.3
5.8
5.5
5.2
5,.0
4.8
4.5
4.3
4.2
4.0
3.8
3.7
3.2
Col"
~e
ll
words exactly, then he was experiencing a form of clozure.
Thus
the name "cloze".
Since then, the cloze procedure has been further
introduced to the f
!~;>Ia ui-
reading ,)y such researchers as Rankin
(1957, 1965), Jenkinson (1957).
Other sp(;Ciallsts such as Manis <1961),
Weaver (1963), Bormuth (1963), and Greene (1964) have studied the use
of the cloze procedure in many varieties of ways to show its usefulness
to education generally and reading specifically.
They have shown the
cloze method Is equal to or better than other existing methods of
testing the comprehension of a reader.
The ease of construction for
cloze exercises should also be taken Into consideration when prepariny tests.
it.
One simply chooses a random passage with about 250 words In
Every fifth word is then deleted from the passage and replaced
with a standard length blank.
blanks in the passage.
the missing words.
This wi I I give approximately fifty
The students are then told to try to fl I I In
Guessing on their part Is encour.aged when necessary.
Whi Je no time limit Is imposed on the students, it was found for an
exercise that It would take the students usually between 15 and 20
minutes to complete the paper.
Bormuth 1 s grading criteria (1967)
was used in which he compared scores on fifty cloze passages with
scores on multiple choice tests over the same materials and found
that a cloze score of 44% was equivalent to a score of 75% on the
multiple choice test.
He also found that a cloze score of 57% was
comparable to answering 90% of the items correctly on a conventional
test.
15
15
sormuth, John "Development of Readability Analyses", u.s.
Department of Health, Education, Welfare, University of Chicago.
March, 1969. p. s-4.
12
TABLE It I
A FORMULA FOR PRODUCING READABILITY
DALE-CHALL
Article=----------------
Page No.
Author'--------------------~---------Publisher: ______________ Date a_ _ __
From
To
1.
Number of words in the sample •••••••
2.
Number of sentences in the sample •••
3.
Number of words noton the Dale List.
4.
Average sentence length (divide
1 by 2) ••••••••• ~···················
5.
Dale score (divide 3 by 1, Multiply by 100) o o o a e e a o e-o • e o • o o • • o • • • •
o o
6,
Multiply average sentence length
(4) by .0496 ••••••••••••••••••••••••
7.
Multiply Dale score (5) by .1579 ••••
8.
Constant •••••••••
9,
Formula raw score (add 6,7, and 8).o
0
~
• • • • • • • •
. . . . . . . . . . .
.
Average raw score of samples •••••
Average corrected grade-level. • .,.
a
Dale, Edgar, and Jeanne S. Shall,
Formula for Predicting
Readability, Bureau of Educational Reasearch, Ohio State
University, Educational Research Bulletin, Vol. 27, Feb. 1948.
Chapter 3
METHOD AND MATERIALS
Since the purpose of this study Is to prove the hypothesis
tnat students wI II do better worK when the readab I I i ty of the
materiai Is easy for them rather than average to difficult, It Is
necessary to have the proper materials with v1hlch to evaluate.
The cloze
procedurd has been chosen tor this study, due
to Its ease of aaministratlon as wei I as the fairly simple metnod
of grading and lnterpreti.ng the scores.
DA TA
NEEDED
In order to estaDilsh some sort of conclusion that Is
reliable, certain types of data must be obtained.
First, there must
be sample passages trom at least three different grade levels.
For
this study, sample passages were taken from the fifth grade through
the ninth grade reading levels.
This gave a spread of five years
between the highest and lowest.
The passages were selected at
random from California State Series textbooks which were In each
classroom but which were not likely to have been used so far this
year.
The three readability formulas-- Dale-Chal I, Fry, and
Flesch --as described In the previous chapter, were employed to
check out each passage chosen for the cloze tests.
Ill,
IV~,
and V) ~
(See Table I I,
It was found that the actua I_ grade pI a cement
13
14
TABLE IV
FLESCH FORMULA
Article: ____________________________
Page No.
Author: ___________________________
From
Publisher a_____________ Date:_
To
1.
Number of words in the sample •••.•• o
2.
Number of sentences in the sample ••
3.
Number of syllables in sample ........
4.
Average sentence length (divide
by 2).~···~~··~···~····~~··········
5.
Number of syllables per 100 words
(divide 3 by 1, multiply by 100) •••
6.
Multiply average sentence length
(4) by 1.015 •••••••••••••••••••••••
7.
Multiply number syllables per 100
words ( 5) by • 846, •••.•.•••.•.•••• , ••••
B.
Cnnstant 206.835 •••.••••••••••••••••
9.
Sum of 6 and 7 •••• "' •••.•••••••••.••••
10.
Flesch Reading Ease Score ••••••••••
Average raw score of samples •••••
Average corrected grade-level ••••
Flesch, Rudolph,
and Row, 1949-
The Art Qf Readable Writing, Harper
FLESCH READING EASE SCALE
Reading
Ease
Score
Description
of Style
Syllables
per 100
words
Average
Sentence
Lengtn
Grade
90 to lUO
Very easy
123
8
5th
80 to 90
Easy
131
11
6th
70 to 80
Fairly easy
139
14
7th
60 to 70
Standard
ll.f'(
17
dth and 9th
50 to 60
Fairly difficult
155
21
lOth to 12th
(high school)
30 to 50
Difficult
167
25
13th to 16th
(college)
Very difficult
192
29
College graduate
0 to 30
-I
):>
(D
r
rn
<
U1
16
levels ranged between grade level 4.3 and 8.3.
This still was a
spread of four years so the study was
as planned.
contln~ed
Also needed for the proper Interpretation of the cloze
scores would be the I. Q. grades of the four classes to be tested.
SOURCES OF DATA
The passages developed for the cloze technique were taken
from the actual classrooms to be tested.
The books were selected
by the classroom teacher on the basis that none of the students
shou I d have read any of them.
A sample passage was then picked at
random within the book.
The students to whom these tests were administered come
from a low-middle-class environment located in a semi-Isolated
rural community.
The town itself contains approximately 300 homes.
The total enrollment of the school is about 180 children including
kindergarten through e.' ghth grade.
These chi I dren are brought to
school by bus from the surrounding area which extends approximately
ten to twelve miles in ai I directions from the school.
To equate
this group wl th a hetero.;Jeneous urban group of students may or may
not be va I I d.
TOOLS AND TECHNIQUES
All of the students In the fifth, sixth, seventh, and
eighth grade classes were administered five
cl
reading material.
dfffer~ent
samples
This material consisted of sample pages from
the fifth, sixth, seventh, eighth, and ninth grade books from the
classrooms.
Each sample passage consisted of approximately 250
17
words.
Using the method of cloze construction advocated by Bormuth 16
every fifth word was deleted In the passage and replaced with a
blank of fifteen spaces.
students to fl I I in.
The result was fifty cloze Items for the
As research by Bormuth has shown
17
there is
little or no difference In results when the exact word Is required
for each cloze Item rather than accepting similar words or synonyms.
Therefore only exact words were counted as being correct,
The f lrst
paper was administered by the author, giving the instructions very
carefully to the class (see CLOZE TEST DIRECTIONS, TABLE VI>.
The
other four tests were administered by the regular classroom instructors at their discretion.
The correct answers were not given to the
teachers to insure that no help would be given the students.
more than one paper was gIven to any c I ass the same day.
No
Due to
normal basences, some days more children took the test than others.
The California Test of Mental Maturity had been administered
to the entire school just one month prior to the cloze testing.
Collecting these results and f indtng the class's average I. Q. was
another method of checking to see if there were any_ great discrepancies
in the data.
16
Bormuth, "New Developments in Readabl lityn, Elementary
Eng I ish, 1967. p. 841.
17
-
1bid.
18
TABLE VI
GLOZE TEST DIRECTIONS
Please say:
I am going to give you a test.
It is important that you
do the best job you can, but the scores will not go on any
school records.
Please clear your desks.
You will receive a booklet.
Please do not turn it over
until I tell you to do so.
When all of the materials are handed out, say:
ttNow turn over the test booklet.
Find the Word
DIRECTIONS.
Read the directions silently while I read them aloud.
THEY SAY ••• This is a new kind of a test.
some short paragraphs.
Words have
b~en
You will read
left out of the
paragraphs and blank spaces were put where the words were
removedo
Your job will be to decide what word was left out of each
space and write that word in the space.
Reading the other
words in the sentence will help you decide what the missing word should be.
It will help you in taking the test if you will remember
these things:
1.
Write only one word in each blank.
2.
Fill every blank.
3.
Skip hard blanks and come back to them later.
4.
Wrong spelling will not count against you if
Don't be afraid to guess.
we can tell what word you meant.
19
TABLE VI -GLOZE TEST DIRECTIONS •• CONT.
S,
Most of the blanks will be filled with ordinary words, but a few will be, •• contractions
or proper names.
6.
Write neatly."
"Let's try a couple of samples to see how it works.*'
Look at Sample
A.*
Christmas.
ttA word is missing from the sentence.
the missing woro is?
(Call on student for reply,)
Write the word in the blank.
(Call on studento)
Look at Sample
B.*
(Merry)
What was the clue word?
(Christmas)
John and Bill go _____.T.o________
Sky Blue Mesa School.
SCHOOL
What do you think
like to go to
THEY
there because the teachers
ARE
so nice.
Fill in the spaces with the wards you think were left out.
How should the paragraph read?
(Call on student.)
"When you have filled in all the blanks, read the story
again to see if it makes sense,
When you have finished,
and checked your answers, go on to the next story."
"Now are there any questions about how you take the test?
During the test I will not be able to answer any questions.
After you have finished, turn your booklet face down on
the desk.
Then take out a book and read,
Please do not
talk while the test is in progress,
You will have plenty of time,
page,
You may begin."
Open the booklet to the first
20
TABLE VI - CLOZE TEST DIRECTIONS •• CONT.
*Cornell, Robert K., Mosberg, Ludwig, and Potter, Thomas c·.
"The Relation Between Gloze and Multiple-Choice Test
Scores As a Function of Relative Paragraph Difficulty and
Grade Level"p SWRL, July 1, 1968.
Chapter 4
ANALYSIS OF DATA
The purpose of this study is to see if there is any basis
for the Idea that when reading material Is presented to students
be Iow thei-r own grade Ieve I, the informatIon gaIn of the students
decreases.
Phrased another way, - at what level should the text-
books for a class be written?
AI I of the effort thus far has gone
into collecting and organizing various data sheets from which answers might be obtained.
In APPENDIX A samples of the original cloze passages can
be found.
These are exact copIes of the papers the students had to
work wl th.
In ADPENDI X B are graphs showing the t nd l vi dua I scores
of each of the students on each of the five sample papers.
21
22
TABLE VIll
AVERAGE SCORES ON CLOZE TESTS
PASSAGE
CLASS
E
D
F
H
G
5
38.6
28.9
20.8
13.6
10.7
6
62.7
48.3
34.5
26.9
20.0
7
57.1
48.0
44.9
35.4
31.3
8
62. I
46.4
39.7
28.7
26. I
On pass.age D, the fIfth grade c Iass has a spread of scores
ranging from two to 62, which is fairly evenly distributed.
average score was 38.6.
56.
The
On passage E the scores range from 2 up to
Here the average. grade is 28. 9.
(See Tab Ie l X, X).
The same
group on Passage F ranges from 4 to 54 with an average of 20.8.
Passage G shows the fifth grade class starting to scew (bunch) new
the lower end of the scale.
(See Table XI 1).
0 to 42 with an average of 13.6.
The grades range from
When we look at Sample H <Table XI I 1),
we find almost alI of the scores between 0 and tO although the highest one is at 42. The average for Passage H is 10.7. Bormuth's 18
studies showed that for students able to answer less than 25% of the
cloze items there was very I ittle Information gain.
Between 25% and
35% an increase was noted and thus his conclusions that 55% on a
cloze readability test seems to represent a criterion for deciding
18
Bormuth, £E._ .£Li• 1969; p. s-5.
TABLE IX
COMPOSITE SCORES
ON
PASSAGE D
AV5RAGE
38,6
80
AVERAGE
•
AVERAGE
56.7
62.7
..
"
• " " "
70
J;rl
ll::
60
•
•
t
• •
" •
" • "
••
•
"
b
{/)
•
•
•
50
\..!)
•
• •
<t:
•
•
E-t
z
• " •
• •
0
J;rl
• •
" •
" •
• •
(I)
40
f.:r.1
62.1
• •
•
•
•
•
• •
• ••
•
•
•
••
•
•
• •
• •
•
•
•
•
..
•
u
•
AVERAGE
•
~
J;rl
tlt
30
•
fzl
•
•
N
0
H
0
•
20
•
10
•
•
•
•
0--~~~_.--~~~_.~~~~~--~~~~
FIFTH
SIXTH
C L AS S E S
SEVENTH
EIGHTH
P AR T I C I P AT I NG
23
TABLE X
COMPOSITE SCORES
ON
PASSAGE E
AVERAGE
AVERAGE
AVERAGE
48.0
48.3
28.9
AVERAGE
46.4
7C
• • •
0
•
•
•
•
Cl)
60·
1%.1
• •
• •
•
•
•
•
•
•
•
t
~
• •
#
~
•
0
u
Ul
• •
•
5(
•
•
f
C!J
<
•
•
rxl
f-!
• •
4(
• •
'
4
• •
•
•
z
•
f:l
•
• •
u
¢.
rxl
•
3(
• •
• •
•
•
•
.
&lof
1%.1
N
0
•
2(
•
t-1
u
•
•
1<,
•
•
0
..
•
SIXTH
FIFTH
c
LASS E S
24
SEVENTH
P AR T I
EIGHTH
c I PAT I
N G
TABLE XI
COMPOSITE IDF SCORES
ON
PASSAGE F
AVERAGE
34.5
AVERAGE
20.8
AVERAGE
39.7
AVERAGE
44.9
70
• ..
t:ll
•
60
(l.1
p::
0
u
t:ll
50
~
C!l
• •
<!!
t
E-t
40
•
•
z
w
u
'
•
•
•
•
•
(\
'
•
t
30
t
'
• •
• •
p:l
w
• • '
• •
~
•
•
•
• •
p.,
• •
•
ttl
' •
lO
0
•
•
,.
N
20
• •
'
~
u
•
•
•
•
• •
10.
0
•
•
•
' •
•
•
FIFTH
C LAS
SIXTH
SEVENTH
s E S PART
25
1
c
EIGHTH
1 PAT 1 N G
TABLE XI I
COHPOSITE SCORES
ON
PASSAGE G
AVERAGE
AVERAGE
13.6
26.9
AVERAGE
AVERAGE
35.4
28,7
•
60
, •
Cl)
111
'
tr:
0
.
50
(.)
•
•
Cl)
111
G
40
•
•
'
• ..
<
z
30
~
'
•
•
•
~
II
•
•
i
0
111
0
114
•
0
0
•
~
f.i1
•
'
E-t
(.)
'
• •
•
•
•
•
• •
•
'
•
•
•
t
111
20
• •
'
N
0
t-l
'
• •
.
•
•
(.)
10
•
0
•
•
0
•
•
'
'
•
• •
II
f)
•
•
"
• •
FIFTH
SIXTH
c LA'S s E S
26
SEVENTH
p ART
I c I
EIGHTH
PAT I N G
TABLE XIII
COMPOSITE SCORE
ON
PASSAGE H
AVERAGE
26.1
AVERAGE
AVERAGE
20.0
AVERAGE
10.7
31.3
70·
•
(I)
•
60·
"
1%1
p::
•
0
•
t
(.)
(I)
50
" •
fzl
•
t'J
<t'l
f-1
"
t
4C
z
•
(.)
•
• •
•
•
..
•
•
3(.
ll-1
•
.. •
" •
r.zl
2C.
'
•
•
...:I
•
(.)
. •
•
• •
• •
~
1(
•
•
• • •
• •
• •
• •
0
..
•
• .
•
N
0
•
•
•
I>
~
1%1
"
6
..
1%1
..
II
•
FIFTH
•
.
•
•
• •
• •
0
• •
•
f
0
•
t
.. "
•
SEVENTH
SIXTH
c LAS s
27
E
s
EIGHTH
PART I c I PAT I N G
28
whether or not students are able to exhibit a maximum gain atter
reading the passage.
The resulTs show that only on Passage D did
the fifth grade exceed this 35%.
(Table IX).
The sixth graae on Passage D ranged from .58% to 84% while
the average was 62. I.
On Passage E the range was from ts% to 68%
with an average of 48.3.
On Passage F the scores ranged from 12 to
48% with an average or 54.J.
AI 1 three of these passages seem to be
within the grade level of the 6th grade.
The sixth grade on Passage
G average on I, 26. ':J wh I Ie ra.ng I ng between 6% and 64fl.
H the range dropped from 4% to 50%.
Here the
ave~age
On Passage
was 26. 1%.
<Table XIII).
Grade 7 ranged trom 14% to 70% on Passage D while averaging
only 56.7.
On Level E the 7th grade ranged from IL% ro
averaged only t.t8.0.
from 16:f, to 76%.
68~
but
On Passage F they aver.aged 44.9 with a range
On Passage G the 7th grade ranged rrom 16% to
56% wh i Ie averagIng 55.4.
Leve I
H.
gave them a range from 2 to 66;b
wltn an average of 31.3.
The eighth grade scored 62.1% on PassageD and ranged trom
8% to 82%.
70%.
39.7.
On
Passa~e t
tney averaged 46.4% and ranged from 2% to
On level F the tlth grade ra.nged from 10% to 60% while averaging
The average on level G was 2tl.7
from 6% t..:.; 50%.
(Table XII) while rangin8
On level H the 8th grade aver,ged L6.1;; while
ranging from 6% to 50%.
In trying to analyze the results (see Table VI I I, p. 22)
one can see that as the difficulty of the passage increased, the
grades for each class decreased.
With the exception of Passage D,
the seventh grade consistently scored higher than the eighth.
When
29
TABLE XIV
AVERAGE CLASS SCORES FOR
grades 5, 6, 7, 8 on
THE CALIFORNIA TEST OF MENTAL
MATURITY
s·
7
_.)
LU
>
LU
_J
~6
<(
_J
0
5
96
90
95
100
MENTAL MATURITY SCORES
105
30
The scores of the
c.
T. M. M. are studied <Table XIV), it can be
seen that the average I. Q. of the se.venth grade rates 100 wht le
the 8th grade average I. Q. Is only 93.
the results.
This would seem to explain
The sixth grade class I. Q. is 101 whl lethe 7th
grade Is 100.
There is not enough difference between the two to
19
warrant any consideration. In reading of Schneyer 1 s study
where
he compared cloze tests to scores on the California Test of Mental
Maturity, it Is learned that he found I. Q. related signif lcantly to
cloze scores.
The cloze scores recorded for each grade for the various
levels may be changed into corrected scores by use of Bormuth's
eq u iva Ient score tab Ie shown be Iovl: 20
TABLE XV
EQUIVALENT CLOZE AND MULTIPLE-CHOICE TEST
PERCENTAGE SCORES*
Cloze Test
Scores
Multiple-Choice Test Scores
Raw
Corrected
19
23
27
31
50
55
60
65
33
40
47
53
35
38
42
46
70
75
80
85
60
67
73
80
50
53
57
90
95
!00
87
93
100
*The standard error of the b or ~egression coefficient was .037.
19Potter, S?_ ill· , p. 13.
20 sormuth, Journal .2f. Reading, 10:291-99, 1967, p.295-6.
31
The standard error of this regression on the multiplechoice scores was 6 percentage points.
Bormuth asks the reader
to keep in mInd that the raw scores are on mutt ip I e choice tests
and therefore comparabl:e scores, when using the table, must be
multiple choice scores.
He has formulated a corrected score which
would be very similar to scores obtained had corrp letion tests
been used.
Bormuth cautions the potential user, however, that
since there was a tendency for the multiple-choice scores to be
skewed, with a number of scores pi ling up at the upper extreme,
this indicates that the multiple-choice tests underestimate the
Thus the completion tests
abi I itles of the more able subjects.
would score somewhat higher In the upper ranges also.
Another
factor that must always be kept in mind is that multmple-shoice
tests vary a great deal from one test writer to another.
Table XVI shows the results of comparing the cloze test
scores of this study with the corrected scores in Bormuth's chart.
TABLE XVI
CLASS
SCORES COW ARED
TO BORMUTH"S EQUIVALENT SCORES
GLOZE SCORES
GRADE
0
E
F
G
CORRECTED SCORES
0
H
E
F
-
G
H
5
38.6
~8.9
20.8 13.6
10.7
67
50
36
6
62 .. 7
48.3
34.5 26.9
L0.6
100
.83
60
47
.$4
7
':J6.7
48.0
44.9 35.4
5 I. 5
99
85
17
60
';Jj
8
b2.l
46.4
59.7 213.7
26.1
100
80
70
50
46
32
In studying the corrected scores in Table XVI, it should be
noted that every grade has lower scores as the sample passages become more difficult.
Whl lethe eighth grade does rank below the
seventh most consistently, this does not happen for Passage D.
would Indicate that Passage D is much too easy for them.
This
Again,
Bormutn cautions anyone using his charts that the reliabl lity depends
heavl ly upon the l8ngth of clo4e tes used and the number of subjects
given the test.
Bormuth used the following furmula to estimate the standard
error of the mea,, of a c Ioze test contaIning .!l l terns and given to N
subjects: 21
S.E. <n,N) =
~ y.t.l3bn
n
+ .561) 2 + n(m- n)
N
m
Solving this equation for various comoinatlons
yielded the results shown in Table XVI I.
error is directly related to the
scores.
The size
01
n and N
of the experimental
size of the standard deviation of
The sizes of the standard deviation of scores are in turn de-
pendent upon the homogeniety of the subjects in a sample.
of the standard deviations
difficulties of the tests.
O'
close test scores are influenced by the
The standard deviations of the tests In
Bormuth's study were plotted against the means.
21
The sizes
Bormuth's plot shows
John Bormuth, "Optimum Sample Size and Cloze Test Length
in REadaollity Measurement," Journal of Educational Measurement.L
( J une, 1965) , p • I 13.
33
that for tests with means below
2~
percent, there Is a tendency for
the standard deviation to Increase with the mean.
This Is undoubtedly
due to the relatively higher frequency of minimum scores.
Using Bormuth's table, "Average Standard Error Calculated for
Various Combinations of Numbers of Items (n) and Numbers of Subjects (N)"
as found In the Journal ,2!_ Educational Measurement, 2 , June, 1965,
page I 14,
and figurIng an average of 84 subjects takt ng each of
tlve cloze tests containing approximately 50 cloze items, the
standard error of
between 2.9 and 3.0 Is reached.
Chapter 5
DISCUSSION
Our problem, as stated previously, was to try to find out
whether the question posed by Both Bormuth and Coleman was val,i'd or
nor.
They stated that as material was presented to a group of stu-
dents below their reading level, the information gain acquired by the
students would decrease.
In comparison to material presented to them
at their own reading level,
their score would be much lower on
materials that they felt were "beneath them."
One of the first things noticed, even In this smal I study,
is that the various test scores do seem to follow the class's
average I. Q., as recorded In Chapter 4.
The average I. Q. of the
eighth grade class is 7 points below that of the others.
Using only
one or two of the students to obtain these I. Q. results could result in the wrong assumption as the standard deviation of this I. Q.
test Is 15 points.
By using the complete class, however, this
author feels that most of the error in measurement would have cancelled Itself out and that the resulting averages are fairly accurate.
More should be done in this area to see exactly what Influence I. Q.
has on cloze scores.
Is there a possibl lity that instead of measuring reading
level, the cloze Instead-- In some way-same factors found In an I. Q. test?
actually measures the
If it should turn out that
34
35
both tests are measuring the same thing, perhaps we need to change
our concept of I. Q. as well as that of readabi I tty level.
Tab Ie VI I I shows that In each grade, 5-th through 8-th, as
the material presented becomes more difficult, the scores dropped
accord! ng Iy
While the formulas <Dale-Chall, Fry, and Flesch)- see
Tab Ie I - rate both Passage F and Passage G a Imost exact Iy the same
level, when we look at the actual test scores of the pup! Is we find
a noticeable drop In the class averages.
This would Indicate that
perhaps something might be wrong with the readability formula results.
As much as possible, the classroom situation remained constant
throughout the testing period.,
The tests were not administered in
ranking order except that all of the classes received Passage OJ
the easiest one, at the beginning.
If just one class dropped on
Passage G, the teacher cou Id be the cause, but when a I I grades
followed this pattern, It Indicates a discrepancy somewhere.
If we use Bormuth's findings that the cloze scores must
average 35% before the students are showing maximum information
gain, we can see that whi ie the fifth and sixth grades fal I exactly
into this pattern (that is, both score at this level when administered
their own grade level material) we find that the seventh grade and
eighth grade results are just reversed.
That is, the seventh grade
average score on the eighth grade material is 35% while the eighth
grade fa I Is below this at the seventh grade level.
The eighth grade class has one student who consistently has a
cloze score of 2 to 4% irregardless of what level the test Is on.
36
Perhaps this one score should be el imlnated to see what would happen
to the other averages.
not felt
For the purposes of this test, however, it was
that any scores should be removed, for fear of Invalidation
of the resu Its.
After analyzing alI of the data collected, It seems apparent
that for the grade levels tested and the materials used, there is a
definite pattern existing.
,;~Tl"'
CONCLUSIONS
Due to the pattern that shows throughout the cloze testing,
it is concludrd that the lower the reading level of the material,
the more information gained by the students.
Thus, if we could
take the Information that an eighth grade class is required to learn
and write It at a fifth grade level, we would have more learning
take place than if we were to present the same material at an
eighth grade Ieve I.
It is also concluded that there is definitely a correlation
between cloze tests and scores on Mental Maturity rests.
Another conclusion f_l lowing this study is that cloze
testing p;ocedures are easier and probably more reliable than the
traditional method of determining readabi I ity.
Cloze tests are
a Iso far more reI i ab Ie than mu It i pIe-choIce tests for measuring
information gain.
This is due to the fact that the students are
not required to read anything previously and there is no way the
teacher can influence the test.
37
IMPLICATIONS FOR FURTHER STUDY
I.
What relationship Is there between cloze test scores
and I. y. scores?
2.
How far would the pattern of higher cloze score
associated with easier material continue?
3.
Would the results of this study be transferable to every
subject in every grade?
That Is, what was the novelty or "Hawthrone"
effect on the class during these tests?
4.
Would the results be the same If our passage had been
Iarger and if the subjects had been from a Iarge urban center?
5.
Would these results hold true if we extended our student
samplings up through high school and col lege level as Coleman
did. 22
6.
What effect, If any, would Illustrations on the sample
passages have on the results?
7.
At what level should classroom textbooks ee written for
23
maximum information gain?
8.
Before textbooks are adopted by the state for classroom
use, could the publishers be forced to give accurate information as
to their readabl lity level.?
22
Coleman, Miller, "A Measure of Information ~ained During
Prose Learning." Readln9 Research Quarterly 2, 1963, p. 369.
·
23
Bormuth, £E. !?.!..!:, 1969 p. s-5.
Ch.;pter 6
SUMMARY
John Bormuth states
24
t;;at a student's Information
that some of his research has shown
wi II decrease as tne reading material
25
becomes easier for him; that Is, below his own reading level, Coleman
~ain
makes the same statement In his researcH.
The purpose of this
study was to prove or disprove thns assumption.
AI I of the children in grades
f~wa 1
six, seven, and eight
in a smal I rural school In Los Angeles County were used as subjects.
They were administered cloze tests made from five different grade
levels of reading samples, ranging from fourth grade level to ninth
grade I eve I ,
The cloze testing technique was used due to its ease of administration as wei I as its comparable ease of grading.
In this pro-
cedure every fifth word was deleted from a 250 word sample andreplaced with a blank of uniform size.
The students were to guess what
words were left out by using various contextual clues found In the
rest of the passage.
26
Bormuth
found that a cloze score of 35% was the minimum
needed for the students to be able to gain maximum knowledge.
24
25
26
Bormuth,
KIare,
~cit.,
~
Bormuth,
2.!.· ,
~cit,
1970, p. 2.
p • 30,
p. s-5.
38
The
39
The results showed that as the reading material became easier, that
is, at a lower grade level, the cloze scores for each of the four
classes tested became higher.
Mental maturity scores for the four
grades tested were plotted and found to vary In direct relationship
with the cloze scores.
The actual reading level (not that shown by
the publishers) was plotted using the DAie-Chal I, Fry, and Flesch
reading formulas.
It was found that the cloze scores para! lei the res
suits of the actual reading level almost exactly.
It Is the conclusion
of this author that the assumption stated in the beginning (quoting
Bormuth) does not hold true, at least for this smal I study.
The
lower the grade level of the reading material, the higher was the
average scores of the classes tested.
If the results of this study hold true in other situations
then perhaps our book publishing companies should begin publishing
books written with a lower readability level so that our students may
gain as much new information from the textbooks as possible.
B I B L l 0 G R AP HY
40
BIBLIOGRAPHY
Bond, Gary Le and Tinker, Miles A. Reading Difficulties,
Their Diagnosis ~ Correction. New York: Appleton-Century
Crofts Inc., 1957.
Bidwell, Corrine Johanna 11 The Readability of Selected Printed
Health Education Materials for the Intermediate Grades",
San Fernando Valley State College, June 1969.
Bormuth, John R. "Gloze Tests as Measures of Readability
and Comprehension Abilitytt, submission for Doctor of Education degree, Indiana University, August, 1962.
Bormuth, John R. ncomparable Cloze and t-iultiple-Choice
Comprehension Test Scores", Journal .Qf. Reading, 1967o
Bormuth, John R. 11 Development of Readability Analysis",
U. Sa Department of Health, Education, and Welfare, office
of Education Bureau of Research, University of Chicago,
Chicago, Illinois, March, 1969. (Project Noo 7-0052, Contract No. OEC-3-7-070052-326).
Bormuth, John R. lfNew Data on Readabilityu, Proceedings
~ International Reading Association, 1968o
£f
Bormuth, John R. "New Developments in Readability", Elementary English, 1967.
Bormuth, John R. "Optimum Sample Size and Gloze Test Length
in Readability Measurementtt, Journal £1 Education Measure~' June, 1965.
Bormuth, John R. "Proposal for Research Submitted to the
SU. c. Commissioner of Education for Support through
Authorization of the Bureau of Research", The University
of Chicago, Chicago, Illinois, September 30, 1970.
Bormuth, John R. ttReadabilitya A New Approach", Reading
Research Quarterly, 1966.
Bormuth, John R. nThe Gloze Readability Procedurett,
Elementry English, 1968.
Chall, Jeanne and Edgar Dale "Familiarity of Selected
Health Terms 11 , Educational Research Bulletin, November 15, 1960.
Chall, Jeanne s. Readability& ~Appraisal 2f Research
and Application, Columbus: Ohio University, 1958e
Coleman, E. B. and Miller, G. R. "A Measure of Information
Gained During Prose Learning", Reading Research Quarterly 3,
1968.
Dale, Edgar, and Jeanne Chall ttThe Concept of Readabilitytt,
Elementry English, January, 1949.
41
Dechant, Edward V. Imnroving
Englewood,Cliffs, New Jersey:
£1 Readin&A
Prentice-Hall, Inco, 1964,
~Teaching
Flesch, Rudolph 11 A New Readability Yardstick", Journal
of Applied Psychology, June, 1948.
Flesch, Rudolph ~ S2
and Brothers, 1951.
~
Readability, New York:
Harper
Fry, Edward nA Readability Formula That Saves Time 11 ,
Journal 2f Reading, April, 1968.
Fry, Edward Teaching Faster Reading, Cambridge, England:
Cambridge at University Press, 1963.
Gray, William s. and Bernice E. Leary "What Makes A Book
Readable? 11 , Journal of Adult Education, October, 1934.
Klare, G. R. "Table for Rapid Determination of Dale-Chall
Readability Scores", Educational Research Bulletin,
February, 1952.
Klare, George R. ~Measurement gf Readability, Ames,
Iowa: Iowa State University Press, 1963.
Lorge, Irving "Reading and Readability", Teachers College
Record, November, 1949.
Potter, Thomas c. ''A Taxonomy of Cloze Research, Partla
Readability and Reading Cemprehension", Southwest Regional
Laboratory for Educational Research and Development, Inglewood, C~lifornia, 1968.
Rankin, E. F. ttResearch design and the Cloze Procedure",
Proceedings of the International Reading Association ~
yention, 1966, z-(1 •
Rankin, E. F. Residual gain as a measure of individual
differences in reading improvement. Journal £! Reading,
1965.
Rankin, E• F. "The Cloze Procedure--A Survey of Research",
In E. L. Thurston and L. E. Hafner (Eds.) ~ Philosophical
~ Sociological Bases 2f Reading, Fourteenth Yearbook of the
National Reading Conference, Milwaukee, Wisconsin, The
National Reading Conference, Inc., 1965.
Spache, G. "A new readability formula for primary grape
Materials:, Elementry School Journal, March, 1953.
Spache, George D. ~Reading~~ Readers, The
Garrard Press, Champaign, Illinios, 1958.
Stone, Clarence R. "Measuring Difficulty of Primary
Reading Material: A Constructive Criticism of Spache's
Measure", Elementary School Journal, 57, October, 1956.
42
Thorndike, E. L. "Reading and Reasonings A Study of
I>fistakes in Paragraph Reading". Journal of Educational
Psychology~ 1917.
AP P E ND I X
A
43
APPENDIX A
GLOZE TESTS
PASSAGE D
t~en
thing
Clara was eleven
HAPPENED
the top of a
•
Little Clara took care
OF
his forehead.
She
WHATEVER
After
ILL
•
her brother.
ached, she put cool
HEAD
FROM
and hurt himself.
accident he grew very
his
old, a sad
Her brother David fell
ROOF
HIS
YEARS
WAS
When
CLOTHS
on
not afraid to do
hard things the doctor
ASKED
her to.
Week after
DAVID
WEEK
lay sick.
and month after month
"Stay with
Clara, u he begged.
uyour
You
the only one who
ARE
feel better.n
So
EVER
ME
are so gentle.
H..A.NDS
FOR
,
MAKES
me
two years Clara hardly
left her brother's side.
-------I~T~------ was not until Clara
FINALLY
thirteen that a doctor
CURE
----~~--------
David.
David was well at
NOT
----~~--------
found a way to
Clara was happy
LAST
know what to do
•
WAS
THAT
But now she did
WITH
herself.
Clara's mother looked _____A.T._________ her youngest daughter
with
A
worried expression.
44
11
I 1 m sorry _ __..T...H_A_T.._____ David wanted you to
CLARA
him all the time,
NURSE
Barton said.
"It
FEW
only gained a
Mrs.
too hard for you.
WAS
are much too pale.
YOU
tt
'
you've
AND
pounds in the last
TWO
years."
"I don't care
DO
be.
------~I.________
now.
--~I•'-M._
it was hard," Clara
"I wanted to nurse _ _....D-..A-.v....I_n...____ •
_ .....A..N=S....
W=E=RE;;;;;.;;D-......_.
trouble is that
IF
------~--------
The
don't know what to
I don't know
---------------WF~T
I want to
FOR
_________ really not much good
anything."
TO
"You're good enough
brother's life,"
save your
father said with a
HER
_ ___,s_M_I....L_E..___ _ •
Source: TRAIL BLAZERS Q[ AMERICAN HISTORY, Ginn and Company,
1961. uclara Barton", p. 226 - 227.
PASSAGE E
Hardly stopping to unload
WORK
John set to
with his ax.
THE
fell nicely, just above
OTHER
ONLY
last creek, he was
WAS
There was no
thought
JOHN
THE
It was
home, and his heart
set on getting there _____T~HA--T~---- night.
Besides, he had
case, he
TREE
The
a moment.
ALMOST
pack,
water.
tree close by, but
about that for
crossings.
HIS
no trouble on the
HAD
He was sure-footed,
HAD
AND
run across on only
OTHER
in every
------~0-N~E~-----
45
of the logs.
He ----~G~A~V-E~------ the tree a kick,
SUDDENLY
it lay steady, and
MIND
BUT
he made up his
to forget what Seth _____.HA~·~D=--==--- said.
ge~ ----~O~V~E·R~=====-
He could
THE
easily enough and see
Judge that evening.
With
HIS
------~--------
AND
ax in one hand
he stepped out on
AS
HIS
furs on his back,
his gun in the
-------T~H~E~-----
log.
--~0-T~HE~R~--
It felt solid
stone under his feet, ----~A~N~D~----- he
went along at _________A._-=--- steady pace.
-------cO=F.______ water just under the
----~A~N~D~----------
his eyes were on
side already, and
TIME
_______ meant
----~B~AR~K
_____.H_I~S._______ head was quite clear,
nothing to John.
------~--------
The race
----~T-HE.________
other
thought that , in his
HE
LOT
, he had crossed a
of logs
more shaky _______r_HA_._N______ this one.
It was _____.J_u_s~T.______ when· he was halfway
OVER
that the log rolled, ____W~I~T-H=O-U~T---- any
warning, and pitched
The
WATER
JOHN
into the creek.
took hold of him
---------------AND
pulled him straight down _____A_N_n_________ rolled him over and
OVER
like a dead pig.
chance even _______
T~O._
----~H;E~------- had no
______ yell.
Sourcer WINGS TO ADVENTURE, Ginn and Company, 1961, "Two
Logs Crossing",p. 205 - 207.
PASSAGE F
My Uncle Fred lives _____;I~N________ Utah.
When the
46
WEATHER
cool
begins to creep down
the mountains, he starts
TO
FRQ~
-----=~--------
think about hunting.
He _____c.L.E~A_N_s______ his rifles, polishes his
and repairs his camping
EQUIPMENT
SHOTGUN
He digests whole
•
books ____A_B=O~U-T_______ hunting for deer, tracking
lions, and cornering wild _....;;.P.:I.;::G.:;S_ _ _ _ •
manufacturers of
questing
--~HU~N~T~I~N~G~----
INFORMATION
a successful hunt.
He
MOUNTAIN
He bombards
equipment with letters re-
about new devices that GUARANTEE
__u_N_T_s________ all the local
--~HA
SPORTING goods stores.
Last year Uncle
FRED
prepared with more
than ___u~s_u~A~L~----- care for his annual _ _.....HU
..· ;;,;:N;.;.;T:::.-----·
He tracked down special
wagon wtth enough equipment
ON
safari
NIGHT
OPENED
hills for four days.
the evening of the
SIGHT
What would he have
would he tell?
EYES
of the hunter's
the back of the _____w_A_G~O-N______ ?
ADVENTURE
THE
and stayed up in
IN
caravan .......;;R-..E__TU=R=N=IN...,G...__.
•
before the
day, our eager, searching
were rewarded by the
--~~~--------
MONTH
the move for a
hunting seas.on
STATION
keep an African
TO
Uncle Fred left the
FOURTH
for his various
outfitted himself and his
AND
rifles
AMMUNITION
IN
What tales of high
G_R_E_A_T_______ dangers
What ____
_____w_E._________ rushed into the driveway
had he overcome?
surrounded Uncle Fred.
AND
Solemnly,
GATE
HE
let down the tail
and undid the folds
OF
a
47
tarpaulin.
A
Instead of
·branching
or a wild pig
HORNS
silent
sharp tusks, ther lay
UNCLE
a jackrabbit.
noble deer with
WITH
__________ death
--~I~N
Fred had finished his
HUNT.
Source: OUR LANGUAGE TODAY 7, California State Series,
American BOOk Company, New York, 1966, p. 151 a 152
PASSAGE G
There was, once upon -------A~------- time in the
VILLAGE
little
owned a flock of
of Guadalupe, a farmer
--~C~H~I-C~K~E~N-S.____ •
The chickens were plump
.. unusually content.
AND
because the farmer, a
YOUNG
Manuel lived on
A
CHICKENS
There they found many
succulemt
yard.
----~W~O~RM~S~-----'
GOOD
THE
run free in the
HOME
and woods around his
FIELDS
CONTENT
very edge of
THE
village and he let his
They were
man named Manuel, did
coop them up in
NOT
----~WH~O~------
•
things to eat
---
crunchy insects, perfectly
shaped ____s...T...o...N_E_s...___ to f i 11 the crop _ _. .,; ,;AN;; ; ,;,; oo_ _ __ aid
digestion.
Small wonder
the chickens were
THAT
a _____.w_E.L.L._______ adjusted and satisfied flock.
__-as_MA._L_L._______ wonder that they greeted
sunrise with loud exclamations
There was one
----~T~H~A~T~-----
satisfied.
----~Y~o-u.N.G.______
delight.
of the flock, however,
was neither well adjusted
He was a
----~W~A~S~-------
MEMBER
OF
EACH
NOR
and gawky rooster who
experiencing the joys and
SORROWS
of
48
growing up.
He
_.::::L::l~S..:,;T.::::L:.:::E~S.::::S.::::L:.::Y__
followed the rest of
flock through the fields,
THE
BUT
usually he would stray ____..:,;F~R~O~M~·------- the others.
he
away because of the
WANDERED
OLDER
received from the
--~PE~C~K~I-N_G_s~--
THOSE
to be one of
who just cannot
_....:C-=O~N.::..F.;;;O.:.:RM::.:-_ _ •
individuals
----~W~H~I~L=E~----~
the young rooster
R_A_r_s_E______ his wings, beat them
would _____
and utter a strange
he
roosters or because he
-~H~A::..P::..PE=N~E=:D::;;....,__
Every once in a
Perhaps
--~T-=O~G.::::E..:,;T~HE~R~--·
almost like the
NOISE
squeak ----~O~F~·--------- a rusty hinge.
HIS
Alas&
had not yet found
voice was changing, and
HE
-----MI~T~s~·-------
__
T_E_____ , when
Sad to _____R_E_LA
real power.
IN
he did succeed
emitting a fairly respectable
____,;C:;.;:R~O;;.:W:.;._.___ , no one gave him ____AN..; ; .;. Y: . - - - - recognition.
As a result,
OF
felt a deep sense
HE
frustration.
Source: OUR LANGUAGE TODAY, California State Series, American
Book Company, New York, 1966, "The Rooster That Couldn't
Tell Timett, P• 203 - 204
PASSAGE H
Snow was already falling,
only the
HAD
---------------LAST
----~~--------
than usual.
and
OUR
week in October.
All trails
AND
it was
Winter
EARLIER
already set in, a month
roads were covered,
only guide post was the
SUMMIT
----~w_oAu_L~D~----
never reach.
which it seemed we probably
Despair drove
ALTHOUGH
MANY
nearly frantic.
Each family
49
TRIED
to negotiate the mountain passes
found it impossible to
ACCEPTED
DO
so.
When it was
that the wagons could
dragged through the
provisions
PASSES
WERE
UP
NOT
be
, their supplies and
packed on oxen and
attempt was made, men
AND
ANOTHER
women walking in snow
to their waists, carrying
children in their arms,
their
CATTLE
•
The Indians said they
ahead with the
if we kept pushing
BECOME
ON
BEGINNING
FTNALLY
OF
could not take
so the others who
NIGHT
SUMMIT
was so full of
OBJECTS
UNDER
KNEW
might bring yielded to
WITHIN
three
•
That night came the
FLAKES
PREVIOUS
THEY
many, and we camped
campfires
be
our party were so
and exhausted with the
the danger that the
miles of the
HE
a forced march until the
side of the summit
STEP
THE
ACROSS
, but that it would
labor that they declared
another
and
that we could get
reached, but some members
TIRED
GUIDES
impossible if snow began.
was in favor of
OPPOSITE
COULD
halt was called, and
REPORTED
came back and
THEIR
trying to drive
A
WENT
Stanton
AND
so
find no road;
BUT
DREADED
snow.
Around the
the trees great feathery
came whirling down.
THEM
The
_________
-----A~I~R
that one could see
only a few feet _____A~W-A~Y_______ •
50
Sourcez OUTLOOKS THROUGH LITERATURE, Scott, Foresman and
Company, 1968, Glenview, Illinois, "Death at Donner Lake"
by Virginia Reed dMurphy. p. 189.
AP P E ND I X
B
51
PASSAGE D
FIFTH GRADE STUDENTS
AVERAGE SCORE 38.6
70
• •
•
60
•
(/)
t
ILl
~
0
u
50
(/)
•
t
l'zl
(!)
-<.
40
E-1
z
ILl
u
~
30
•
Tx1
p.,
•
111
N
20
0
....:!
u
10
•
II
0
•
5
10
NUM B E R
15
0 F
S T U
20
DE NT S
52
PASSAGE D
SIXTH GRADE STUDENTS
AVERAGE SCORE 62.7
80_
•
•
•
•
It
70
•
•
• •
(I)
f.il
~
60
0
C)
•
(I)
•
3
f.il
N
0
1-1
C)
2
1
0~---------------------------------20
15
10
5
N U M B' E R 0 F
S TUDE N T S
53
PASSAGE D
SEVENTH GRADE STUDENTS
AVERAGE SCORE 56.7
80
70
'
• • •
.
.,
60
t
•
• • •
5
• •
4
3
txl
N
0
1-1
2
0
10
0 ,...___ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
~·
5
10
NUMBE R 0 F
15
S TUDE N T S
20
54
PASSA<GE D
EIGHTH GRADE STUDENTS
AVERAGE SCORE 62.1
.
"
8
• •
7
'
f
'
•
• •
Ul
6
rx.1
p::
0
• •
u
Ul
• •
5
rx.1
C!J
<(
~
4
~
rx.1
u
0..:
rx.1
3
Ill
rx.1
N
0
2
1-1
u
1.
•
0
~------------------------------------------------25
20
15
10
5
NU MB E R 0 F
S TUDE NT S
55
GRAPHIC RESULTS
ON
PASSAGE D
62.1
8
(.!)
z
.....
E-1
<
fl.!
.....
7
(.)
.....
E-1
p:;
<
flt
Cl)
~
62.7
6
Cl)
Cl)
<
...;J
(.)
5
0
10
20
30
40
50
CLOZE PERCENTAGE SCORES
60
70
56
PASSAGE E
FIFTH GRADE STUDENTS
AVERAGE SCORE 28.9
.:.80
70
{I)
~
t:t::
60
• •
0
u
{I)
~
50
CJ
<
• •
E-f
z
• •
40
•
~
u
t:t::
~
30
P.4
f
f,ll
N
0
r-l
20
u
,
1
• •
•
0
5
NUM B E R
10
0 F
15
S T U D E N T S
20
57
PASSAGE E
SIXTH GRADE STUDENTS
AVERAGE SCORE 48.3
70
..
..
•
•
60
• •
0
0
50
..
'
• •
(J)
•
•
.,
fil
N
2
•
0
t-ot
•
(.)
1
•
5
10
15
20
NUMB E R 0 F S T U D E N T S
58
PASSAGE E
SEVENTH GRADE STUDENTS
AVERAGE SCORE 48.0
t
70
• • •
60
..
(/)
r:.:l
• •
~
0
•
{) 50 .
•
(/)
.
•
•
•
t
.
•
• •
•
rxl
N
0
20.
1-1
{)
10
•
0~------------------------------------------25
20
15
10
5
NUMBER
OF
STUDENTS
59
PASSAGE E
EIGHTH GRADE STUDENTS
AVERAGE SCORE 46.4
{/}
Jxl
60.
• • •
• •
• •
p::
0
(.)
{/}
50
•
M
•
(!)
• •
<
~
z
40
Jxl
• •
(.)
p::
f:tl
At
30
Jxl
N
0
1-1
20
(.)
10
OL-------~~------------------------------5
10
15
20
NUM!fER
OF
STUDENTS
60
GRAPHIC RESULTS
ON
PASSAGE E
8
CD
z
I-
<
(L
7
48.0
6
48.3
u
I0::::
<
(L
U)
w
U)
U)
<
_J
0
28.9
5
10
20
30
40
50
CLOZE PERCENTAGE SCORES
60
70
61
PASSAGE F
FIFTH GRADE STUDENTS
AVERAGE SCORE 20.8
70
til
60
fit
~
0
u
til
50
f-11
~
<
E-t
40
z
M
u
~
f.il
3
t:li
• •
• • •
rz:l
N
0
H
•
2
• • •
u
•
• •
1
•
•
0
. ..
•
#
10
5
N U M
B~
E R
0 F
15
20
s TUDENTS
25
62
PASSAGE F
SIXTH GRADE STUDENTS
AVERAGE SCORE 34.5
70
60
Cl}
~
p:';
0
50
u
.
Cl}
..
.
.
,
, "
• •
•
11
~
20
N
"
0
1-1
u
10
OL---------------------------------~
20
15
10
5
NU MB E R
0 F
S T U DE N T S
63
PASSAGE F
SEVENTH GRADE STUDENTS
AVERAGE SCORE 44.9
70
(/)
60
•
111
~
0
u
50
(/)
,
.
• •
• •
111
N
0
20
1-1
•
u
10
•
0~------~------~--------~------~
S
J.O
15
20
NUMB--:ER
OF
STUDENTS
64
PASSAGE F
EimHTH GRADE STUDENTS
AVERAGE SCORE 39.7
•
70
• •
.
60 .
(/)
..
f;El
p::;
50
...
0
u
.
•
(/)
P:1
40
'
C!)
<
•
.
•
E-f
z
M
30.
u
p::;
"
'
ttl
t).,
20
N
'
0
~
() 10
0~----~~------~------~------~~-NUMBER
OF
STUDENTS
65
GRAPHIC RESULTS
ON
PASSAGE F-
39.7
8
C!)
z
1-f
E-l
44.9
7
<
A..
1-f
(.)
H
E-l
~
<
p.,
6
U)
r:zl
Ul
Ul
<t!
.....:1
0
20.8
5
10
20
30
40
50
60
70
66
PASSAGE G
FIFTH GRADE STUDENTS
AVERAGE SCORE 13.6
60
U)
w 5
p::
0
u
•
U)
4
w
(.!J
<tl
•
E-t
z
w
•
3
-
u
p::
f,r.l
p.,
2 .
•
• •
f,r.l
'
N
0
...:1
•
1
•
u
..
•
• '
• •
• •
5
NUMB<ER
10
OF
15
STUDENTS
20
67
PASSAGE G
SIXTH GRADE STUDENTS
AVERAGE SCORE 26.9
•
60
(J)
f.xl
p::
~0
0
..
u
Cl)
f.xl
40
CJ
<I!
•
E-t
z
f.xl
•
l 0
u
•
tx:
f.xl
P-4
• •
a.o
• •
f.xl
• • •
N
0
~
u
10
•
•
••
0~------~------------------------------20
15
5
10
NUMBER
OF
STUDENTS
68
PASSAGE G
SEVENTH GRADE STUDENTS
AVERAGE SCORE 35.4
60
•
Cll
•
•
50
f£1
p::
0
• •
u
Cll
•
•
40
•
rq
•
Cl
<
~
•
30
z
rq
•
u
• •
•
•
•
•
!l::
~
20
AI
•
•
1£1
N
0
10
...:1
u
0~--------------~------~------~~~
5
10
15
20
NU MB E R 0 F S T U D E N T S
69
PASSAGE G
EIGHTH GRADE STUDENTS
AVERAGE SCORE 28.7
60
Ul
f,zl
~
•
so
•
0
u
.. .
Ul
•
•
• •
•
• •
•
•
2
•
1 •
•
..
•
•
5
N U M B-- E R
10
0 F
15
S T U D E N T S
20
70
GRAPHIC RESULTS
ON
PASSAGE G
8
C!J
z
H
E-t
7
35.4
<
Il-l
H
(.)
H
E-t
~
<
Il-l
6
(/}
fil
fJ)
(/}
<
1-l
b
5
CLOZE PERCENTAGE SCORES
71
PASSAGE H
FIFTH GRADE STUDENTS
AVERAGE SCORE 10.7
60
C/}
fz.l
n:: 50
0
u
(I)
72
PASSAGE H
SIXTH GRADE STUDENTS
AVERAGE SCORE 20.0
70
•
9-0
..
(/)
w
p::
50.
0
u
(/)
M
40
Cl
<
E-t
z
w 30
u
•
0::
, •
f:t.1
At
,
20 .
w
~
N
0
•
...:1
u
10
•
..
.. ..
.. •
. •
(I
,
0
. •
5
NUMB E R
15
10
0 F
s
20
T U D E N T S
25
73
PASSAGE H
SEVENTH GRADE STUDENTS
AVERAGE SCORE 31.3
70
"
C>
60
"
Ul
M
•
~
0
(.)
•
50
0
Ul
,
• •
.
3
•
..
p
M
N
2
0
H
.
.
f
(.)
0
1
.,
• •
,
0~------------------------~------~--------~-5
10
15
20
25
NUMBER
OF
STUDENTS
74
PASSAGE H
EIGHTH GRADE STUDENTS
AVERAGE SCORE 26.1
70
Ul
60
..
~
.
p:;
0
(.)
,
so
• •
Cl)
•
f:rl
e,
4Q
•
<
•
E-t
z
~
•
•
30
(.)
~
f:rl
p.,
0
•
•
20"
• •
f:rl
• •
N
0
...:I
10
0
•
(.)
~
tJ
•
10
5
N U M
BE R 0 F
15
20
S TUDE
NT
25
s
75
GRAPHIC RESULTS
ON
PASSAGE H
26.1
8
(.!)
z
H
H
3-L,3
7
<
p.,
H
(.)
H
H
IX:
<
p.,
6
{/)
I':LI
{/)
{/)
<
1-1
u
5
0
10
20
30
40
50
GLOZE PERCENTAGE SCORES
60
70