2/16/2011
Curriculum-Based Measurement
for Written Expression (CBM-W):
The “How To” Guide
Kristen McMaster, Ph.D.
University of Minnesota
Curriculum-Based Measurement
for Written Expression (CBM-W)
1. Select
materials
5. Monitor
progress
2. Administer
CBM-W
4. Graph
Data
CEHD
3. Score
CBM-W
College of Education + Human Development
1
2/16/2011
Step 1: Select Materials
•
Select type of measure based on:
•
•
•
•
Grade level
Purpose of progress monitoring
What will provide YOU with the most meaningful information.
Determine the number of prompts needed.
•
•
•
How many weeks do you plan to monitor progress?
How often do you plan to monitor progress?
Create prompts that…
•
•
•
•
Tap background knowledge that your students are likely to have.
End mid-sentence (if narrative/story starters).
Include a “tell why” component (if expository).
Print prompts at top of pages that students will write on.
Tip
Keep materials and procedures
consistent throughout the
Monitoring period.
”If you want to measure change,
don’t change the measure!”
3
CEHD
College of Education + Human Development
Sample Prompts
•
•
•
•
•
•
•
•
•
•
•
Narrative (Story Starters)
One night I had a strange dream about…
It was the last day of school so I decided to…
One day, when I got home from school…
One day my friend told me the strangest story…
I was walking home when I found a $100 bill on
the sidewalk and…
One day I went to school but nobody was there
except me, so I…
I was talking to my friends when, all of a
sudden…
One day I woke up and was invisible and…
One day I found the most interesting thing and…
One summer I went on a trip and…
I was walking
lki down
d
the
th street
t t when
h I saw...
CEHD
•
•
•
•
•
•
•
Expository
Describe a game you like to play and tell why
you like it.
Describe your favorite day of the week and tell
why you like it.
D
Describe
ib your favorite
f
i time
i off the
h year andd tell
ll
why you like it.
Describe your favorite thing to do and tell why
you like it.
Describe a place you like to go and tell why you
like to go there.
Describe the clothes that kids wear in your
school and tell why kids wear them.
Describe the friends you have and tell why they
are your friends.
4
College of Education + Human Development
2
2/16/2011
Sources for Prompts
• Research Institute on Progress Monitoring
http://www.progressmonitoring.org
• Write Source
http://www.thewritesource.com
• NWREL
http://www.nwrel.org/assessment/pdfGeneral/Prompts_BlowingAway.pdf
5
CEHD
College of Education + Human Development
Prompts for Beginning Writers
• Words and sentences were drawn from Minneapolis first-grade
language arts curriculum (Houghton-Mifflin).
• These reflect high-frequency words used with early elementary
students.
• Prompts are available on RIPM website:
http://www.progressmonitoring.org
6
CEHD
College of Education + Human Development
3
2/16/2011
Step 2. Administer CBM-W
• Read prompt to students
• Give students 30 seconds to think
• Reread prompt
• Give students X minutes to write (choose time
based on grade level, and stay consistent all
year)
Tip
See handouts; beginning
writing procedures are on
RIPM website.
7
CEHD
College of Education + Human Development
Score CBM-W: Procedures
• Words Written (WW):
– The number of words written in the sample.
– A “word” is defined as a sequence of letters separated
by a space from another sequence of letters.
8
CEHD
College of Education + Human Development
4
2/16/2011
Score CBM-W: Procedures
• Words Spelled Correctly (WSC):
– The number of correctly spelled words written in the
sample.
– In some studies, words correct are counted regardless of
whether they are used correctly within the context of the
sentence (e.g., “I saw a bare when I was camping.”), and
i others,
in
h
only
l when
h the
h words
d are usedd correctly
l within
i hi
the context.
9
CEHD
College of Education + Human Development
Score CBM-W: Procedures (cont.)
• Correct word sequences (CWS) (Videen et al., 1982): The number of
adjacent, correctly spelled words that are syntactically and
semantically
ti ll correctt within
ithi the
th context
t t off the
th sample.
l
“Capitalization and punctuation are taken into account.”
Example:
“^The^man^saw^a bare in^the^tree^and begn to^run^.”
CWS=9.
10
CEHD
College of Education + Human Development
5
2/16/2011
Score CBM-W: Procedures (cont.)
• Correct minus incorrect word sequences (CIWS) (Espin et al.,
1999). The number of incorrect word sequences subtracted from the
number correct sequences.
q
Example:
“^The^man^saw^avbarevin^the^tree^and vbegnvto^run^.”
CWS=9.
11
CEHD
College of Education + Human Development
Score CBM-W: Procedures (cont.)
• Correct minus incorrect word sequences (CIWS) (Espin et al.,
1999). The number of incorrect word sequences subtracted from the
number correct sequences.
q
Example:
“^The^man^saw^avbarevin^the^tree^and vbegnvto^run^.”
CWS=9.
IWS=4.
12
CEHD
College of Education + Human Development
6
2/16/2011
Score CBM-W: Procedures (cont.)
• Correct minus incorrect word sequences (CIWS; Espin et al., 1999).
The number of incorrect word sequences subtracted from the number
correct sequences
sequences.
Example:
“^The^man^saw^avbarevin^the^tree^and vbegnvto^run^.”
CWS=9.
IWS=4.
CIWS=5.
13
CEHD
College of Education + Human Development
Graphing Data
•
Baseline: Collect 2-3 samples. Graph the scores.
Example:
•
Baseline Day 1 = 25 CWS
•
Baseline Day 2 = 24 CWS
•
Baseline Day 3 = 23 CWS
14
CEHD
College of Education + Human Development
7
2/16/2011
Graph Baseline Scores
15
CEHD
College of Education + Human Development
Graphing Data (cont.)
•
Baseline: Collect 2-3 samples. Graph the scores.
•
Draw a vertical line to separate the baseline data from the first phase
of instruction.
16
CEHD
College of Education + Human Development
8
2/16/2011
Graph Baseline Scores
17
CEHD
College of Education + Human Development
Graphing Data (cont.)
•
Baseline: Collect 2-3 samples. Graph the scores.
•
Draw a vertical line to separate the baseline data from the first phase
of instruction.
•
Draw a goal line corresponding to the desired weekly increase for
the total number of weeks of instruction.
18
CEHD
College of Education + Human Development
9
2/16/2011
Setting the Goal
• There are no set guidelines for determining
desired rate of progress.
• You might wish to use:
– Published norms (e.g., AIMS website
http://www.aimsweb.com)
– Local school or district benchmarks
– Classwide data
Tip
Set reasonable/attainable
yet ambitious goals.
19
CEHD
College of Education + Human Development
Setting the Goal
•
Determine the following:
b: Student’s baseline performance
Example: b = 24 CWS (median of 25, 24, 23)
m: Desired rate of progress
E
Example:
l m = 1.5
1 5 CWS per weekk
x: Amount of time the student will be monitored
Example: x = 35 weeks
•
Solve for y (the student’s goal):
y = mx + b
Example:
y = (1.5 CWS per wk * 35 weeks) + 24 CWS
y = 52.5 + 24 = 76.5 = goal in 35 weeks
20
CEHD
College of Education + Human Development
10
2/16/2011
Draw the Goal Line
76.5
21
CEHD
College of Education + Human Development
Setting the Goal: Practice
• b = baseline: 27 CWS, 32 CWS, 22 CWS
– Use median or mean
• Median (middle score) = 27
• Mean (27
(2 + 32 + 22)/3 = 27
2
• m = Desired rate of progress: 2 CWS per week
• x = Monitoring period: 20 weeks
• Goal: y = mx + b =
22
CEHD
College of Education + Human Development
11
2/16/2011
Setting the Goal: Practice
• b = baseline: 27 CWS, 32 CWS, 22 CWS
Use median or mean
• Median (middle score) = 27
• Mean (27
(2 + 32 + 22)/3 = 27
2
• m = Desired rate of progress: 2 CWS per week
• x = Monitoring period: 20 weeks
• Goal: y = mx + b = (2 * 20) + 27 = 67
23
CEHD
College of Education + Human Development
Case study: Annie
Annie is a 16-year old tenth grader who has yet to pass the
statewide graduation test in writing.
She has been placed in a standards English class in an
effort to help her pass the high-stakes tests.
Her English teacher, Mrs. Penn, decides to monitor her
writing progress using CBM.
Mrs. Penn will use this progress monitoring data to decide
whether her instruction is effective.
24
CEHD
College of Education + Human Development
12
2/16/2011
Annie
• Which type of writing prompt would you select?
• For how many minutes would you administer each probe?
• What scoring procedure would you use?
25
CEHD
College of Education + Human Development
Annie
• Which type of writing prompt would you select? Narrative
• For how many minutes would you administer each probe? 7-10
• What scoring procedure would you use? CWS or CIWS
26
CEHD
College of Education + Human Development
13
2/16/2011
Annie’s Writing Sample
The cave was very dark. I try to close my eyes, so I couldn’t see
anything but that didn
anything,
didn’tt help.
help Than I hear some one breathing.
breathing I try to
stream, but nother came out. The breathing became close and close to me,
and the worst Part was that I couldn’t see athing. At first I thought meself
that I am Just emaging stuff.
27
CEHD
College of Education + Human Development
Scoring Practice: Words Written
The cave was very dark. I try to close my eyes, so I couldn’t see
anything,
y
g, but that didn’t help.
p Than I hear some one breathing.
g I tryy to
stream, but nother came out. The breathing became close and close to me,
and the worst Part was that I couldn’t see athing. At first I thought meself
that I an Just emaging stuff.
Words written = 58
28
CEHD
College of Education + Human Development
14
2/16/2011
Scoring Practice: WSC
The cave was very dark. I try to close my eyes, so I couldn’t see
anything,
y
g, but that didn’t help.
p Than I hear some one breathing.
g I tryy to
stream, but nother came out. The breathing became close and close to me,
and the worst Part was that I couldn’t see athing. At first I thought meself
that I an Just emaging stuff.
Words written = 58
Words spelled correctly = 54
29
CEHD
College of Education + Human Development
Scoring Practice: WSC
The cave was very dark. I try to close my eyes, so I couldn’t see
anything, but that didn’t help. Than I hear some one breathing. I try to
stream but nother came out.
stream,
out The breathing became close and close to me,
me
and the worst Part was that I couldn’t see athing. At first I thought meself
that I an Just emaging stuff.
Words written = 58
Words
W
d spelled
ll d correctly
tl = 54
OR 43
30
CEHD
College of Education + Human Development
15
2/16/2011
CWS and CIWS:
Mark the end of each sentence
The cave was veryy dark. I tryy to close myy eyes,
y , so I couldn’t see
anything, but that didn’t help. Than I hear some one breathing. I try to
stream, but nother came out. The breathing became close and close to me,
and the worst Part was that I couldn’t see athing. At first I thought meself
that I an Just emaging stuff.
31
CEHD
College of Education + Human Development
CWS and CIWS: Mark the end of each
sentence.
The cave was very dark. I try to close my eyes, so I couldn’t see
anything,
y
g but that didn’t help.
p Than I hear some one breathing.
g I tryy to
stream, but nother came out. The breathing became close and close to me,
and the worst Part was that I couldn’t see athing. At first I thought meself
that I an Just emaging stuff.
32
CEHD
College of Education + Human Development
16
2/16/2011
Score incorrect word sequences first.
The cave was very dark. I  try  to close my eyes, so I couldn’t see
anything, but that didn’t help.  Than  I  hear  some one  breathing. I 
try  to  stream , but  nother  came out. The breathing became  close
 and  close  to me, and the worst Part was that I couldn’t see athing .
At first I thought  meself  that I an  Just  emaging  stuff.
33
CEHD
College of Education + Human Development
Then score correct and incorrect word
sequences.
The cave was very dark. ˆ I try to ˆ closeˆmyˆeyes,ˆsoˆIˆcouldn’t ˆsee ˆ
anything, ˆbut ˆthat ˆdidn’t ˆhelp ˆ.  Than I  hearsomeone breathing ˆ.
ˆI  try  to  stream , but nother came ˆout ˆ. ˆThe ˆ breathing  became
close and close  to ˆ me,ˆand ˆthe ˆworst ˆPart ˆwas ˆthat ˆI ˆcouldn’t
ˆseeathing . ˆAt ˆfirst ˆI ˆthought  meself that ˆIan Just emaging
stuff ˆ.
Correct = 36
Incorrect = 27
Correct minus incorrect = 9
34
CEHD
College of Education + Human Development
17
2/16/2011
Case Study: Annie’s Graph
CBM Writing - Annie
40
Correct Minus Incorrect Sequences
Baselin
30
20
10
0
Dates of Probes
CEHD
College of Education + Human Development
Case Study: Annie’s Graph
CBM Writing - Annie
40
Correct Minus Incorrect Sequences
Baselin
30
20
10
0
Dates of Probes
CEHD
College of Education + Human Development
18
2/16/2011
Case Study: Annie’s Graph
CBM Writing - Annie
40
Correct Minus Incorrect Sequences
Baseline
30
20
10
0
Dates of Probes
CEHD
College of Education + Human Development
Case Study: Annie’s Graph
CBM Writing - Annie
40
Correct Minus Incorrect Sequences
Baseline
Direct Instruction
30
20
10
0
Dates of Probes
CEHD
College of Education + Human Development
19
2/16/2011
Case Study: Annie’s Graph
CBM Writing - Annie
40
Correct Minus Incorrect Sequences
Baseline
Direct Instruction
Intervention 1
30
20
10
0
Dates of Probes
CEHD
College of Education + Human Development
Case Study: Annie’s Graph
CBM Writing - Annie
40
Correct Minus Incorrect Sequences
Baseline
Direct Instruction
Intervention
Intervention 2
30
20
10
0
Dates of Probes
CEHD
College of Education + Human Development
20
2/16/2011
Questions for Discussion
• Writing is a complex, multi-dimensional process. How might you
account for changes in the quality of student writing?
• How might you use CBM-W to monitor your own students’ progress
and to make instructional decisions?
CEHD
College of Education + Human Development
References
Espin, C. A., Scierka, B. J., Skare, S., & Halverson, N. (1999). Criterionrelated validity of curriculum-based measures in writing for secondary
school students. Reading and Writing Quarterly: Overcoming
Learningg Difficulties,
ff
, 15,, 5-27.
Videen, J., Marston, D., & Deno, S. L. (1982). Correct Word Sequences:
A Valid Indicator of Proficiency in Written Expression (Vol. IRLDRR-84). University of Minnesota, Institute for Research on Learning
Disabilities.
CEHD
College of Education + Human Development
21