11/21/2012
Alternate Route to Certification
Introduction to Assessment
of Student Learning
Objectives
• Identify key questions about assessment of student
learning
• Recognize the difference between standardized tests
and classroom assessments
• Recognize advantages and disadvantages of the major
categories of assessments
• Recognize the characteristics of an effective test or quiz
question
• Identify the characteristics of effective rubrics to evaluate
performance tasks
• Recognize the format and content of state assessments
(CMT, CAPT, SBAC)
Agenda
•
•
•
•
•
•
•
•
•
•
Overview of Assessment
Types of Assessment
Break
Creating Effective Assessment Questions
Lunch
Discussion about Rubrics
State Testing
Break
Teacher Accountability
Framework for Effective Assessment Planning and
Design
1
11/21/2012
Pop Quiz
1. For each equation, choose
true or false.
2. Explain how the results of the
race would change if the race used
a clock that rounded to the nearest
tenth
“What students learn depends
as much on your tests as your
teaching.”
WJ McKeachie
University of Michigan
Philosophy of Assessment
Based on your experience as a student, what would
you say is the purpose of assessment?
CCT Assessment for Learning (Domain 5)
• Teachers use multiple measures to analyze student
performance and to inform planning and instruction.
How does your experience compare to the purpose
implied within the CCT expectations for
assessment?
2
11/21/2012
Assessment
The Skillful Teacher
Grading
(Saphier, et al, 2008)
12 Components of Good Classroom Assessment
• Determining the Assessment Task
– Engagement
– Alignment
– Relevance
• Communicating the Standards of Performance
– Transparent and Precise
• Assessing Prior Knowledge before Instruction
– To inform instructional practice
The Skillful Teacher
(Saphier, et al, 2008)
12 Components of Good Classroom Assessment
• Frequent Data Collection
• Frequent High Quality Feedback
– Direct and Specific
– Used by students to improve work
• Student Self-Assessment
• Student Progress Monitoring
3
11/21/2012
The Skillful Teacher (Saphier, et al, 2008)
12 Components of Good Classroom Assessment
• Frequent Error Analysis by Teacher
– Identify patterns of error / misconceptions
– To inform instructional practice
• Error Analysis by Students
• Planning and Reteaching
• Student Goal Setting
• Reporting Systems
Key Questions About Assessment
• Reference pg 2 of the Assessing Student
Learning Handout
• What to assess?
• How much to assess at one time?
• How often to assess?
Types of Assessment
• Formative Assessment
• Interim Assessment
• Summative Assessment
4
11/21/2012
Formative Assessment
• Assessment used to evaluate students’
knowledge and understanding of particular
content; the results are used by the teacher to
adjust and plan instruction to improve
achievement in that particular area
–
–
–
–
Anything that informs instruction
Provides diagnostic information
Occurs prior to or during instruction
Typically short; quick feedback turnaround; not used
for grading purposes
– Integral to teaching-learning process
Formative Assessment
• Find 3 ways to represent the number 53 to
demonstrate your understanding of place value
• Identify one property for bases and one for acids
• Looking at a set of letters on the board, recite
them in random order
Interim Assessment
• Assessment used to evaluate students’ knowledge
and understanding of particular content; the results
are used by the teacher to adjust and plan
instruction to improve achievement in that particular
area
–
–
–
–
–
More formal than formative assessments
Often used for progress monitoring / predictive
Can be analyzed for programmatic purposes
May or may not be used for grading
Positioned widely enough to alter instruction and produce
measureable progress before next assessment
5
11/21/2012
Summative Assessment
• Assessment used to document student
achievement at the end of a unit or course, or to
evaluate the end product of a learning activity or
unit of study
– Occurs after material is taught
– Often used for grading purposes
– Can take many forms (paper and pencil test, paper,
performance, etc.)
Key Questions about Assessment
• What are some different ways to assess?
• How do you assess using performances /
projects?
• How do you reduce subjectivity in assessment?
Variety within Assessment
• Thinking critically and
making judgments
(developing
arguments, reflecting,
evaluating, assessing,
judging)
•
•
•
•
•
•
•
•
•
Essay
Report
Journal
Letter of Advice to .... (about
policy, public health matters .....)
Present a case for an interest
group
Prepare a committee briefing
paper for a specific meeting
Book review (or article) for a
particular journal
Write a newspaper article for a
foreign newspaper
Comment on an article's
theoretical perspective
6
11/21/2012
Variety within Assessment
• Solving problems and
developing plans
(identifying problems,
posing problems,
analyzing data,
designing
experiments, applying
information)
•
•
•
•
Problem scenario
Group Work
Work-based problem
Prepare a committee of
inquiry report
• Draft a research bid to a
realistic brief
• Analyze a case
• Conference paper (or
notes for a conference
paper plus annotated
bibliography)
Variety within Assessment
• Performing procedures
and demonstrating
techniques
(Computation, taking
readings, using
equipment, following
laboratory procedures,
following protocols,
carrying out
instructions)
• Demonstration
• Role Play
• Make a video (write script
and produce/make a video)
• Produce a poster
• Lab report
• Prepare an illustrated
manual on using the
equipment, for a particular
audience
• Observation of real or
simulated professional
practice
Variety within Assessment
• Managing and
developing oneself
• Journal
• Portfolio
• Learning Contract
(Working cooperatively, working
independently, learning
independently, being
self-directed, managing
time, managing tasks,
organizing)
• Group work
7
11/21/2012
Variety within Assessment
• Accessing and
managing information
(Researching,
investigating,
interpreting, organizing
information, reviewing
and paraphrasing
information, collecting
data, searching and
managing information
sources, observing and
interpreting)
• Annotated bibliography
• Project
• Dissertation
• Applied task
• Applied problem
Standardized Assessment
• A test that is
administered, scored,
and interpreted in a
manner
predetermined by the
test producer
• Connecticut Mastery
Test (CMT)
• Connecticut Academic
Performance Test
(CAPT)
• SAT, GRE, LSAT
• AP
• TOEFL
• Stanford-Binet
Types of Standardized Tests
• Aptitude
• Achievement
– Measure of cognitive
ability
– Measure of knowledge
or skills
– Stanford Binet
– SAT I
– GRE
–
–
–
–
SAT II
AP exams
CMT & CAPT
Classroom
assessments
8
11/21/2012
Classroom Assessments
• Pre-Assessment
– Typically formative
– Pre / Post mirrored or pre-requisite knowledge
• Formative Assessment
– Process vs static
• Summative
– Varying formats based on purpose
Characteristics of High Quality
Assessments
Reliable
• CONSISTENT RESULTS ON MULTIPLE ADMINISTRATIONS OR WITH
OTHER EVALUATORS
• CLEARLY FOCUSED ON WHAT IT IS SUPPOSED TO MEASURE
Valid
Fair
Unbiased
• THE NATURE OF THE ASSESSMENT ITSELF DOES NOT PREVENT
STUDENTS FROM ACHIEVING EXCELLENCE
• THE ASSESSMENT DOES NOT OFFEND OR PENALIZE ANY STUDENTS
BECAUSE OF GENDER, ETHNICITY, RACE, SOCIOECONOMIC STATUS, OR
RELIGION
• The type and design of the
assessment should be
determined by the level of
thinking you wish your student
to demonstrate.
9
11/21/2012
Bloom’s Taxonomy
Bloom’s Taxonomy [1956 ] &
Bloom’s Cognitive Process Dimensions [2001]
Knowledge -- Define, duplicate, label,
list, name, order, recognize, relate, recall
Remember Retrieve knowledge from
long-term memory, recognize, recall,
locate, identify
Comprehension -- Classify, describe,
discuss, explain, express, identify,
indicate, locate, recognize, report, review,
select, translate
Understand -- Construct meaning, clarify,
paraphrase, represent, translate,
illustrate, give examples, classify,
categorize, summarize, generalize,
predict…
Application -- Apply, choose,
demonstrate, dramatize, employ,
illustrate, interpret, practice, write
Apply -- Carry out or use a procedure in a
given situation; carry out or use /apply to
Analysis -- Analyze, appraise, explain
calculate, categorize, compare, criticize,
discriminate, examine
Analyze -- Break into constituent parts,
Synthesis -- Rearrange, assemble,
collect, compose, create, design, develop,
formulate, manage, write
Evaluate -- Make judgments based on
criteria, check, detect
inconsistencies/fallacies, critique
Evaluation -- Appraise, argue, assess,
choose, compare, defend, estimate,
explain, judge, predict, rate, core, select,
support, value
Create -- Put elements together to form a
coherent whole, reorganize elements into
new patterns/ structures
an unfamiliar task
determine how parts relate
Webb’s Depth of Knowledge
• DOK-1 – Recall & Reproduction - Recall of a fact, term, principle,
concept, or perform a routine procedure
• DOK-2 - Basic Application of Skills/Concepts - Use of
information, conceptual knowledge, select appropriate procedures
for a task, two or more steps with decision points along the way,
routine problems, organize/display data, interpret/use simple graphs
• DOK-3 - Strategic Thinking - Requires reasoning, developing a
plan or sequence of steps to approach problem; requires some
decision making and justification; abstract, complex, or non-routine;
often more than one possible answer
• DOK-4 - Extended Thinking - An investigation or application to real
world; requires time to research, problem solve, and process
multiple conditions of the problem or task; non-routine
manipulations, across disciplines/content areas/multiple sources
10
11/21/2012
DoK is about complexity, not
difficulty.
• While verbs may appear to point to a DOK level, it is
what comes after the verb that is the best indicator of the
rigor/DOK level.
– Describe the process of measuring to the nearest unit
– Describe how two characters are alike and different; describe an
observation you made about these materials
– Describe the evidence that supports your solution or
conclusions, using words, data, diagrams, etc.
– Describe the evidence you found in 2 or more texts that shows
different perspectives on this topic; describe the most significant
effect of WWII on the nations of Europe
• Depth of Knowlege – Hess Matrix
Hess Cognitive Rigor Matrix
Depth +
Thinking
Level 1
Recall &
Reproduction
Remember
- Recall, locate basic
facts, details, events
Understand
- Select appropriate
words to use when
intended meaning is
clearly evident
- Specify, explain
relationships
- summarize
– identify main ideas
- Explain, generalize,
or connect ideas
using supporting
evidence (quote,
example…)
- Explain how
concepts or ideas
specifically relate to
other content
domains or concepts
Apply
- Use language
structure (pre/suffix)
or word relationships
(synonym/antonym)
to determine meaning
– Use context to
identify meaning of
word
- Obtain and interpret
information using
text features
- Use concepts to
solve non-routine
problems
- Devise an approach
among many
alternatives to
research a novel
problem
Analyze
- Identify whether
information is
contained in a graph,
table, text feature,
etc.
– Compare literary
elements, terms,
facts, events
– analyze format,
organization, & text
structures
- Analyze or interpret
author’s craft
(literary devices,
viewpoint, or
potential bias) to
critique a text
– Analyze multiple
sources
- Analyze
complex/abstract
themes
– Cite evidence and
develop a logical
argument for
conjectures
- Evaluate relevancy,
accuracy, &
completeness of
information
- Synthesize
information within
one source or text
- Synthesize
information across
multiple sources or
texts
Level 2
Skills &
Concepts
Evaluate
Create
- Brainstorm ideas
about a topic
- Generate
conjectures based on
observations or prior
knowledge
Level 3
Strategic
Thinking/
Reasoning
Level 4
Extended
Thinking
11
11/21/2012
Practice w/ Hess CRM
• Consider the fairy tale, Little Red Riding Hood.
– Write a basic comprehension question to assess
students’ understanding of the story.
– Consider where you would place your question on the
Hess CRM.
– Write a more rigorous question (higher cognitive
demand).
– Be prepared to share your examples.
Depth +
Thinking
Remember
Understand
Level 1
Recall &
Reproduction
Who are the characters?
What was the story’s
setting?
Level 4
Extended Thinking
Retell or summarize the
story in your own words.
Identify words/phrases
that helped you to know
the sequence of events
in the story.
Is this a realistic or fantasy
story?
Compare the wolf
character to the
character of Red. How
are they alike-different?
Is this a realistic or
fantasy story? Justify
your interpretation using
text evidence.
Are all wolves (in literature)
like the wolf in this story?
Support your response
using evidence from this
and other texts.
What is your opinion
about the intelligence of
the wolf? Justify your
opinion using text
evidence.
Evaluate
Create
Level 3
Strategic Thinking/
Reasoning
What color was Red’s
cape?
Who is this story
about?
Apply
Analyze
Level 2
Skills & Concepts
Write a telephone
conversation between
Red and her mother to
explain the wolf incident.
Take a 15 minute break!
12
11/21/2012
Type of Assessment Prompts
• Selected Response
– Select correct answer from response provided
• Multiple Choice
• True / False
• Matching
• Constructed Response
– Produce an answer in response to a question
• Fill in the blank
• Short answer
• Essay
Type of Assessment Prompts
• Performance Task
– Demonstrate new learning by completing a task
• Performing or presenting
• Experiment or Demonstration
• Creating
• Portfolio Assessment
– A collection of samples of student work to
demonstrate learning
Activity
– Creating Effective Assessment Questions
• Refer to pg 6 of your handout
• Separate into groups of 4
• Within groups of 4, pair off and jigsaw the
problems with one pair working evens and the
other working odds
• You will have 15 minutes to work in pairs and 10
minutes to report out within your group of 4.
13
11/21/2012
Tips
Refer to pg 11 of your handout
Tips
Refer to pg 10 of your handout
14
11/21/2012
Tips
Refer to pg 15 of your handout
ELA sample
Grade 11
Key Details
Read this sentence from the passage.
• “Besides being beautiful to contemplate, space
diamonds teach us important lessons about natural
processes going on in the universe, and suggest new
ways that diamonds can be created here on Earth.”
• Explain how information learned from space diamonds
can help scientists make diamonds on Earth. Use
evidence from the passage to support your answer.
Closure
Read this sentence from the opening of our
presentation.
• What students learn depends as much on your
tests as your teaching.
• Explain how information learned from this
presentation exemplifies the sentiment of this
statement. Provide examples from the
presentation to support your thinking.
15
11/21/2012
• Break for Lunch
• Reconvene after 1 hour at 12:30
No Child Left Behind
It is impossible to determine
teaching effectiveness without
determining learning results.
Rubrics
A TOOL FOR ASSESSING THE QUALITY
OF A STUDENT GENERATED
PERFORMANCE OR PRODUCT CREATED IN
RESPONSE TO AN ASSIGNED TASK
16
11/21/2012
Rubric
ABOVE
EXPECTED
LEVEL
AT
EXPECTED
LEVEL
BELOW
EXPECTED
LEVEL
Made contact Made contact Rarely or
EYE
with most of with some of never looked
CONTACT the audience the audience up from notes
VOLUME
Could be
heard by
Could be
everyone and heard by
used volume everyone
for emphasis
Hard to hear
3 Types of Rubrics
• Analytical
– Identify and assess components of a finished product
• Holistic
– Assess student work as a whole
• Scoring or Grading
– Used to delineate consistent criteria for grading
Sample Holistic Rubric
17
11/21/2012
Activity
Specific Skill
Language &
Mechanics
Identify two
strengths
and two
weaknesses
of the
sample
analytical
rubric
Content
Brevity
Clarity
Highly Effective
Adequate
Inadequate
Precise, standard
words & sentences; no
errors
Well organized with
standard words &
sentences; only one or
two non-distracting
errors
Hard to follow; awkward;
non-standard English;
many misspellings; too
many errors distract
Strong
Adequate
Weak
One page that includes
all the necessary
details and only the
necessary details
One page with no more
than two unneeded
details
More than two
unneeded details and/or
runs longer than
one page
Easy to follow
Pretty easy to follow
Confusing
Standard memo format
Standard memo format
Nonstandard memo
format
Neat and wordprocessed
Neat but handwritten
Format
Neatness
Class
Requirements
Class
Participation
On time with proper
heading
Actively participated in
the class discussion on
writing effective memos
Messy
Late and/or improper
heading
Moderately involved in
the class discussion on
writing effective memos
Not involved in the class
discussion on writing
effective memos
Effective Rubrics
• Clear, precise and easy to understand
• Observable and measurable criteria
• Focused on key learning
• Reasonable expectations
• Singularity of criterion
• Relevant criteia
Rubric samples
• http://rubistar.4teachers.org/
18
11/21/2012
CAPT Science Holistic Rubric
Analytic Rubric
Music Sample
19
11/21/2012
History Teaching Portfolio Rubric
Integrating Points into Analytic
Rubric
Criteria
K
n
o
w
l
e
d
g
e
Using
historically
accurate
information
Below Level 1
Use of
historical
information
not passable
0
2
4
Level 1
Level 2
Level 3
Limited use of
accurate
historical
information
Accurate for
the most
part; some
important
information
is not
accurate
All information
used is
historically
accurate; most
is relevant to
topic
5
5.5
6
6.5
7
7.5
Level 4
Total
All information used is
historically
accurate and
relevant
8
9
10
Observe how point values are assigned across performance levels.
Grading Rubric Sample
20
11/21/2012
Final Word
• Rubrics design offers significant flexibility.
• Teachers should be prepared to revise their
rubrics multiple times to insure the tool
measures the characteristics they seek to
assess.
• Rubrics are most effective when shared with
students in advance and offered with annotated
student work samples.
• Rubrics can be used in a formative sense to
provide feedback and guidance for revision.
State Assessment
• Overview of the CMT and CAPT
• State results of CMT and CAPT and NCLB
targets
• Smarter Balanced Assessment
• CSDE Performance Index
History of the CMT
• First introduced in 1985
• Now in 4th Generation (since 2006)
• Science was added as a test in grades 5 and 8
in 2008
• Originally developed as a minimum proficiency
examination
21
11/21/2012
Components of the CMT
• Reading Comprehension
–
–
–
–
Forming a general understanding
Developing an interpretation
Making connections to text
Examining content and structure
– 2 sessions @ 45 minutes
– 30-32 questions
– Multiple choice and open ended
Components of the CMT
• Degrees of Reading Power (DRP)
– Ability to understand surface meaning
– 1 sessions @ 45 minutes
– 42-49 questions
– Multiple choice
– Significantly weighted relative to overall Reading
score
Components of the CMT
• Direct Assessment of Writing
–
–
–
–
Narrative – grades 3-4
Expository – grades 5-6
Persuasive – grades 7-8
Tests elaboration, organization, fluency, and
reasoning
– 1 sessions @ 45 minutes
22
11/21/2012
Components of the CMT
• Editing and Revising
–
–
–
–
Capitalization
Punctuation
Editing
Spelling
– Content, organization and tone
– Syntax
– Word choice
– 1 session @ 60 min
– 30-40 questions; multiple choice
Components of the CMT
• Mathematics
–
–
–
–
–
Numerical and Proportional Reasoning
Geometry and Measurement
Working with Data: Probability and Statistics
Algebra and Reasoning: Patterns and Functions
Integrated Understanding
– 25 Content Strands
– 2-3 sessions @ 60 min
– 94-120 questions; multiple choice, open ended and grid in
Components of the CMT
• Science
– Content Knowledge
– Scientific Inquiry
– Life Science
– Physical Science
– Earth Science
– 25 Content Strands
– 1 session @ 65-70 minutes
– 39-48 questions; selected and constructed response
23
11/21/2012
Components of the CAPT
• Response to Literature
–
–
–
–
Basic understanding of text
Interpreting or explaining text
Connecting text to life
Taking a critical stance
– 1 session @ 80 minutes
– 4 open ended questions
Components of the CAPT
• Editing and Revising
– 3 short readings with embedded errors
– 1 session @ 35 minutes
– 18 multiple choice questions
Components of the CAPT
• Reading for Information
– 3 non-fiction articles
– Comprehending non-fiction
– Taking a critical stance
– 1 session @ 60minutes
– 12 multiple choice questions; 6 open ended questions
24
11/21/2012
Components of the CAPT
• Interdisciplinary Writing
– Write persuasively on two topics
– Use support from two readings
–
–
–
–
Taking a clear position
Supporting a position
Organizing ideas
Expressing ideas with clarity and fluency
– 2 session @ 80 minutes each
Components of the CAPT
• Mathematics
–
–
–
–
Algebraic Reasoning
Numerical and Proportional Reasoning
Geometry and Measurement
Probability and Statistics
– 2 session @ 85 minutes each
– 24 Grid In; 8 Open ended
Components of the CAPT
• Science
–
–
–
–
–
Energy transformations
Chemical structures
Global interdependence
Cell chemistry and biotechnology
Genetics, evolution and biodiversity
– Application of scientific knowledge
– Questions related to suggested performance tasks
– 2 session @ 65 minutes each
– 60 Multiple Choice; 5 open ended
25
11/21/2012
2012 State Assessment Data
2012
Math
Science
Reading
Writing
CMT gr5 Prof
85.7%
82.4%
79.7%
82.4%
CMT gr 5 Goal
71.8%
64.1%
67.7%
68.1%
CAPT Prof
78.8%
80.2%
80.9%
88.8%
CAPT Goal
49.3%
47.3%
47.5%
63.1%
Smarter Balanced
September 28,
2012
SBAC Assessment System
English Language Arts and Mathematics, Grades 3 – 8 and High School
Last 12 weeks of year*
DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; an interactive reporting system; scorer training modules; and teacher collaboration tools.
INTERIM ASSESSMENT
INTERIM ASSESSMENT
Computer Adaptive Assessment and Performance Tasks
Computer Adaptive Assessment and Performance Tasks
PERFORMANCE
TASKS
• Reading
• Writing
• Math
COMPUTER
ADAPTIVE ASSESSMENT
Scope, sequence, number, and timing of interim
assessments locally determined
Re‐take option
Optional Interim assessment system —
no stakes
Summative assessment
for accountability
* Time windows may be adjusted based on results from the research agenda and final implementation decisions.
26
11/21/2012
SBAC highlights
 Summative assessment in grades 3-8 and 11
During last 12 weeks of school year
 Computer adaptive testing
Selected response, short constructed response, extended
constructed response, technology enhanced, and
performance tasks
 Performance tasks
Reading
Writing
Math
 Nationally established standard setting
Take a 15 minute break!
Challenges
• Accountability expectations
• SRBI
• Universal Design Principles
27
11/21/2012
Activity
• Design an assessment for your content
discipline
– Work within a small group
– Apply what you have learned
Questions
28