Assessment Workshop
Charlene Stokes
Pam Sharma
Assessment Workshop
Morning Session 8:30 – 12:00 noon
Breakfast
Closing the Loop
Bloom’s Taxonomy of Learning
Student Learning Outcomes
Break
Alignment
Interactive Activity
Lunch
Afternoon Session
Examples of Alignment
Assessment Methods
Break
Interactive Activity
Assessment Report
Assessment Survey
8:30 – 9:00
9:05 – 9:30
9:35 – 10:00
10:05 – 10:30
10: 30 – 10:45
10:50 – 11:20
11:25 – 12:00
12:00 – 1:00 PM
1:00 – 4:00 PM
1:05 – 1:45
1:50 – 2:20
2:25 – 2:45
2:50 – 3:20
3:25 – 3:55
3:55 – 4:00
Assessment
Student Learning
Assessment is a comprehensive concept,
centered on improving student learning
through a variety of ways.
Closing the Loop
Quality Products
Graduates with AA,
AS, AAS degrees
and certificates
Mission
Education
Quality Services
Institutional
Effectiveness and
Research
Support Services
Outcome Assessment
The systematic collection, analysis, evaluation,
and interpretation of information pertaining to
student learning and institutional operations in
order to inform decision making about the ways
in which to improve learning, teaching, and
overall institutional effectiveness
http://iacbe.org/oa-definition.asp
Needs Analysis
•
•
•
•
Master Course Guide
Previously Submitted Assessment Reports
Pre-Assessment Forms submitted spring 2016
Survey Result fall 2016
Survey Results
100
80
Percent Full
time
60
40
Percent
Adjunct
20
0
Components of SLO
100
90
80
70
60
50
40
30
20
10
0
Percent Full time
Percent Adjunct
Assessmetn report
preparation
90
80
70
60
50
40
30
20
10
0
Percent Full time %
Percent Adjunct%
Course Assessment
Student Learning
Course Description
Course
Outcomes
General
Education
Outcomes
Student Learning Outcomes
Goals Objectives Outcomes
Teacher
Centered
I
N
T
E
N
D
E
D
Student
Centered
Goals
Course
Outcomes
Objectives
Student
Learning
Outcomes
O
B
S
E
R
V
E
D
Course Objectives and Learning
Outcomes
The difference between course objectives and
learning outcomes is the former describes an
intended state (what you hope your students
will learn), whereas the latter expresses a
present or observed state (what your students
actually learned).
http://resources.depaul.edu/teaching-commons/teaching-guides/course-design/Pages/courseobjectives-learning-outcomes.aspx
Components of Student Learning
Outcome
Audience
Behavior (action)
Condition
Degree
Bloom’s Taxonomy of learning
Available at: http://1.bp.blogspot.com/-qWrdLWtb04w/TmE8SvMtYUI/AAAAAAAAQ9o/fSl9JcHnPPg/s1600/blooms_taxonomy.jpg
Action Verbs for Bloom’s Learning
Available at :
https://sites.stanford.edu/teachingcommons/sites/default/files/blooms.png
Examples of Student Learning
Outcomes
Art:
Good:
• Students will be able to articulate the role art plays in society using
a written critique of an art work.
• Students will be able to identify the formal elements and principles
of art, which apply to the creation, and discussion of an artwork.
• Students will identify the connection of historical or current events,
which contextualize the making of an artwork.
Poor:
• Students will appreciate art.
• Students will learn how to discuss a work of art.
• Students will be familiar with culture and the relationship of art
making.
Examples of Student Learning
Outcomes
– Good:
– Learners will demonstrate the ability to
communicate effectively in both oral and written
forms. (Univ. of Toledo)
– Poor:
– Students will learn how to effectively communicate
in both oral and written forms. (Univ. of Toledo)
Student Learning Outcomes
Available at : http://www.cobbk12.org/sites/alt/training/Blooms/circle.GIF
Reasons for Bloom’s Taxonomy
• Accurately measuring students’ abilities requires an
understanding of the different levels of cognition that
are critical for learning.
• Developing intended student learning outcomes
according to Bloom’s Taxonomy helps students
understand what is expected of them.
• Plan and deliver appropriate instruction.
• Helps design and implement appropriate assessment
tasks, measures, and instruments.
• Ensure instruction and assessment are appropriately
aligned with the intended outcomes.
Assessment Methods
•
•
•
•
Direct
Indirect
Formative
Summative
Direct Method
Direct methods of assessment ask students to
demonstrate the knowledge or skill.
Some examples of direct assessment are evaluating
students’ abilities to
• summarize a process
• apply a theory
• solve a problem,
• synthesize literature.
• Tests, essays, presentations, etc. are generally
direct methods of assessment.
Indirect Method
A method where students report on their own
learning (reflection). Some examples of indirect
assessment method are
Asking students to rate their knowledge about a
topic after taking the course as compared to
what they believe they knew before taking the
course.
Surveys and interviews are indirect assessment
methods.
Formative Assessment
A regular process of gathering information and feedback
on students' learning as the semester is in progress.
Formative assessment is not graded and is conducted as a
means to collect immediate feedback on whether
students have learned the material or what needs to be
done to improve their learning.
Methods include the mid-semester student surveys,
student focus groups, classroom assessment techniques
such as a minute paper, muddiest points
Summative Assessment
Summative assessments are tests, quizzes, and
other graded course activities used to measure
student performance. They are cumulative and
often reveal what students have learned at the
end of a course. Within a course, summative
assessment includes the system for calculating
individual student grades.
https://www.cte.cornell.edu/teaching-ideas/assessing-student-learning/measuring-studentlearning.html
Assessment and Evaluation
Dimension
Content: timing,
primary purpose
Orientation:
focus of
measurement
Findings: uses
thereof
Assessment
Formative:ongoing, to
improve learning
Process-oriented: how
learning is going
Evaluation
Summative: final, to
gauge quality
Product-oriented:
what’s been learned
Diagnostic: identify
Judgmental: arrive
areas for improvement at an overall
grade/score
Alignment
Student Learning
Outcome
Assessment
Method
Learning
Activities
Assessment
Instrument
Alignment
Student Learning
Outcome
Assessment
Method
Learning
Activities
Assessment
Instrument
Alignment Examples
Type of
learning
objective
Recall
Recognize
Identify
Interpret
Exemplify
Classify
Summarize
Infer
Compare
Explain
Apply
Execute
Implement
Examples of appropriate assessments
Objective test items such as fill-in-the-blank, matching, labeling, or multiplechoice questions that require students to:

recall or recognize terms, facts, and concepts
Activities such as papers, exams, problem sets, class discussions, or concept
maps that require students to:

summarize readings, films, or speeches

compare and contrast two or more theories, events, or processes

classify or categorize cases, elements, or events using established criteria

paraphrase documents or speeches

find or identify examples or illustrations of a concept or principle
Activities such as problem sets, performances, labs, prototyping, or simulations
that require students to:

use procedures to solve or complete familiar or unfamiliar tasks

determine which procedure(s) are most appropriate for a given task
https://www.cmu.edu/teaching/assessment/basics/alignment.html
Alignment Example Contd.
Type of
Learning
Example of appropriate assessment
Objective
Activities such as case studies, critiques, labs, papers, projects, debates, or
Analyze
Differentiate concept maps that require students to:
 discriminate or select relevant and irrelevant parts
Organize
 determine how elements function together
Attribute
 determine bias, values, or underlying intent in presented material
Evaluate
Activities such as journals, diaries, critiques, problem sets, product reviews, or
Check
studies that require students to:
 test, monitor, judge, or critique readings, performances, or products against
Critique
established criteria or standards
Assess
Create
Generate
Activities such as research projects, musical compositions, performances, essays,
business plans, website designs, or set designs that require students to:
Plan
 make, build, design or generate something new
Produce
Design
Putting it all Together
•
•
•
•
Pre-Assessment Form
Pre-Assessment Peer Review
Assessment Report
Assessment Report Peer Review
General Education Outcomes
• Communicate effectively in oral and written
formats
• Employ or utilize information access and literacy
skills
• Demonstrate problem-solving and critical
thinking skills
• Employ mathematical and science literacy skills
• Acquire a cultural, artistic and global perspective
• Demonstrate professional and human relations
skills
WV State Assessment Gen Ed
Communications
One-Year Certificate
• Students will use
appropriate techniques to
deliver messages using
various communication
channels within the chosen
professional environment.
Associate’s Degree
• Students will apply
appropriate techniques to
deliver messages using a
combination of conversing,
speaking, listening,
corresponding, and writing
using various
communication channels.
Alignment
• State outcomes for 1-year and Associate
Degree graduates
• Gen ed outcome--Communication
• Course outcome—that utilizes communication
skills
• Appropriate/matching SLOs
“Student, not teacher driven.”
Teacher
• I will teach my students
what the main themes of
Romeo and Juliet are.
Student
• Students will write a paper
explaining the main themes
of Romeo and Juliet.
Questions to Ask:
1) What should students know and be able to do?
This list of knowledge and skills becomes your . . .
STANDARDS
2) What indicates students have met these standards?
To determine if students have met these standards, you
will design or select relevant . . .
AUTHENTIC TASKS
3) What does good performance on this task look like?
To determine if students have performed well on the task,
you will identify and look for characteristics of good
performance called . . .
CRITERIA
4) How well did the students perform?
To discriminate among student performance
across criteria, you will create a . . .
RUBRIC
5) How well should most
students perform?
The minimum level at which you
would want most students to
perform is your ...
CUT SCORE or BENCHMARK
6) What do students need to
improve upon?
Information from the rubric will
give students feedback and allow
you to ...
ADJUST INSTRUCTION
Guidelines for writing a standard
• Identify what really matters
• Identify what you want the students to take
away from the ….
• Write a standard that is generic and still shows
the development of knowledge and skills of
the students.
• Students will be able to identify themes across
a variety of literature.
Rubric
A coherent set of criteria for students’
work that includes descriptions of
performance quality on the levels.
Different types of Rubrics
Analytic
Holistic
• Definition
• Definition
– Each criterion (dimension,
trait) is evaluated separately
• Advantages
– Diagnostic info: teacher
– Formative info: student
– Easier link to instruction
• Disadvantages
– More time to score
– All criteria (dimensions, traits)
are evaluated simultaneously
• Advantages
– Faster scoring
– Good for summative
• Disadvantages
– Not good for formative
– Overall score doesn’t indicate
areas for improvement
Activity Examples
• Type of performance:
– Products
• Constructed objects
• Written essays, themes,
reports, term papers
– Processes
• Physical skills
• Use of equipment
• Oral communication
• Examples
– Products
• Handmade apron
• Written analysis of the
effects of the Marshall
Plan
– Processes
• Doing a forward roll
• Preparing a slide for the
microscope
• Making a speech to the
class
Activity Examples, cont.
• Bookmark:
– Create a bookmark to match the theme of the story/book.
• Infomercial:
– Students will record a segment that uses persuasion to sell
a product.
• Radio/Newscast:
– Describe a legal case and its outcome.
• Brochure:
– Create a brochure that explains the steps required to
prepare a microscope slide, hookup an IV, or solve a word
problem.
Activity Examples, cont.--flipped
• iClickers/Polling to collect responses to a quiz on
the reading assignment.
• Word Webs/Concept Maps “How are these
concepts, ideas, people, etc. connected?”
• Dyadic Essays Students come to class prepared
with an essay question they have written
concerning the topic/concept, along with the
“idea response.” Students answer each others’
questions, and following the “idea responses” are
shared.
Rubrics used to assess the outcome.
Essential Trait
and
Performance
Level
Organization
Level 3
Excellent
Level 2
Average
Level 1
Poor
Clearly
organized with
headings, intro,
transitions,
body,
conclusion, and
citations
Main idea is
clear, but the
structure is
awkward and
hard to follow.
Ideas are
unfocused or
undeveloped.
Transitions are
nonexistent.
Undeveloped
introduction and
conclusion.
Free of
1-3 errors
More than 3
Activity and the process
Cognitive
Hierarchy
(Trait/Beha
vior)
Sample Learning
Outcome (What)
Sample
Sample
Instructional
Assessment/Activi Strategy
ty (How)
(Method of
preparation)
Analyze
Interpret a company’s
financial statements to
determine credit risk.
Provide students with Case studies, lectures
sample data from a
introducing credit risk
company and assign analysis
an analysis report
within small groups.
Remember
Identify the insertion and
origin of each muscle of the
trunk in the human body.
Label a diagram,
asking students to
identify the origin
and insertion of the
muscles in the
diagram.
Prepare an unlabeled
chart and have
students group
different muscles
together by
origin/insertion
points during a lab
with cadavers.
Understand
Classify common treatments
Working in groups,
Instructor provides
Formative
• Definition: involves activities that measure and
provide feedback during the learning process.
• Formative assessment examples include:
– Providing verbal or written summaries
– Peer activities that reinforce collaboration and assess
the quality of each other’s work
– Self-assessments so learners can evaluate their own
performance
– Lists, charts, and graphic organizers to encourage
learners to communicate clearly and effectively
through making connections and noting relationships
Summative
• Definition: involves using grades, points, or data
to demonstrate learner growth after instruction.
• Summative assessment examples include:
– Performance tasks to see if certain skills were
acquired
– Multiple-choice tests to examine memory recall
– Participation tasks to encourage learners to engage in
lectures, seminars, and online forums
– Oral presentations so learners can demonstrate their
knowledge or present research findings
Emerging
(0-12 points)
1. Organization
(20 points)
Style/Mechanics
(20 points)
Ideas may not be
focused or developed;
the main purpose is
not clear. The
introduction is
undeveloped. Main
points are difficult to
identify. Transitions
may be needed. There
is no conclusion or
may not be clear the
presentation has
concluded. Conclusion
does not tie back to
the introduction.
Audience cannot
understand
presentation because
there is no sequence
of information.
Developing
(13-16 points)
Main idea is evident,
but the organizational
structure many need to
be strengthened; ideas
may not clearly
developed or always
flow smoothly and the
purpose is not clearly
stated. The
introduction may not
be well developed.
Main points are not
clear. Transitions may
be awkward.
Supporting material
may lack in
development. The
conclusion may need
additional
development.
Audience has difficulty
understanding the
presentation because
the sequence of
information is unclear.
Language choices may Language used is
be limited, peppered mostly respectful or
with slang or jargon,
inoffensive. Language is
too complex, or too
appropriate, but word
dull. Language is
choices are not
questionable or
particularly vivid or
Advanced
(17-20 points)
Ideas are clearly
organized, developed,
and supported to
achieve a purpose; the
purpose is clear. The
introduction gets the
attention of the
audience and clearly
states the specific
purpose of the speech.
Main points are clear
and organized
effectively. The
conclusion is satisfying
and relates back to
introduction. (If the
purpose of the
presentation is to
persuade, there is a
clear action step
identified and an overt
call to action.)
Language is familiar to
the audience,
appropriate for the
setting, and free of
bias; the presenter
may “code-switch”
Score
Task Description: Each student will make a 5-minute presentation on the changes in
one Portland community over the past 30 years. The student may focus the
presentation in any way s/he wishes, but there needs to be a thesis of some sort, not
just a chronological exposition. The presentation should include appropriate
photographs, maps, graphs, and other visual aids for the audience.
Knowledge
Presentation skills
Excellent (5)
The presentation
demonstrates a depth of
historical understanding
by using relevant and
accurate detail to
support the student’s
thesis.
Research is thorough and
goes beyond what was
presented in class or in
the assigned texts.
Competent (3)
The presentation uses
knowledge which is
generally accurate with
only minor inaccuracies,
and which is generally
relevant to the student’s
thesis.
Research is adequate but
does not go much
beyond what was
presented in class or in
the assigned text.
Needs Work (1)
The presentation uses
little relevant or accurate
information, not even
that which was
presented in class or in
the assigned texts.
Little or no research is
apparent.
The presenter speaks
clearly and loudly
enough to be heard,
using eye contact, a
lively tone, gestures, and
body language to engage
the audience.
The presenter speaks
clearly and loudly
enough to be heard, but
tends to drone and/or
fails to use eye contact,
gestures, and body
language consistently or
effectively at times.
The presenter cannot be
heard and/or speaks so
unclearly that s/he
cannot be understood.
There is no attempt to
engage the audience
through eye contact,
gestures, or body
General Education Outcome:
Students will prepare an outline organizing their
speech using good grammar.
Essential
Learning
Assessment Results
Trait and
Activity
Instrument
Performance
or Method
Level
12/15 (80%) at
Student
Level 5
prepares an
2/15 (13%)at
outline
of
Organization
Rubric
Level 3
the speech.
1/15 (7%) at
Level 1
Student
Style/Mecha prepares and
outline of
nics
the speech.
Rubric
13/15 (87%) at
Level 5
2/15 (13%) at
Level 3
References
•
The Educational Value of Course-level Learning Objectives/Outcomes
http://www.cmu.edu/teaching/resources/Teaching/CourseDesign/Objectives/CourseLearningObjectivesValue.p
df
•
Common Assessment Terms-Teaching Excellence & Educational Innovation - Carnegie Mellon University
https://www.cmu.edu/teaching/assessment/basics/glossary.html
•
Align Assessments, Objectives, Instructional Strategies-Teaching Excellence & Educational Innovation - Carnegie
Mellon University
https://www.cmu.edu/teaching/assessment/assesslearning/tools/groupselfassessmenttool.pdf
•
https://www.cmu.edu/teaching/assessment/basics/alignment.html
•
Assessment Tools
https://www.cmu.edu/teaching/assessment/assesslearning/tools/groupselfassessmenttool.pdf
•
How to develop Rubrics
https://facultycenter.stonybrook.edu/sites/facultycenter.stonybrook.edu/files/basicpages/91/files/rubric_information.pdf
•
Summative Assessment
https://www.cte.cornell.edu/teaching-ideas/assessing-student-learning/using-rubrics.html
•
Rubric Component Guide
http://iacbe.org/pdf/rubric-component-guide.pdf
References
• http://www.elearninglearning.com/activities/assessme
nt/examples/?open-article-id=4410553&articletitle=formative-vs--summative-assessment&blogdomain=learndash.com&blog-title=learndash
• http://www.aacu.org/sites/default/files/files/publicati
ons/LevelsOfAssessment.pdf
• https://uwaterloo.ca/centre-for-teachingexcellence/teaching-resources/teaching-tips/lecturingand-presenting/delivery/class-activities-andassessment-flipped-classroom
• http://www.teachhub.com/40-alternativeassessments-learning
References
• https://uwaterloo.ca/centre-for-teachingexcellence/teaching-resources/teaching-tips/assessingstudents/aligning-outcomes-assessments-and-instruction
• http://jfmueller.faculty.noctrl.edu/toolbox/standards.htm
• http://jfmueller.faculty.noctrl.edu/toolbox/standardtypes.h
tm
• http://jfmueller.faculty.noctrl.edu/toolbox/workshopstanda
rd.htm
• http://jfmueller.faculty.noctrl.edu/toolbox/workshoprubric.
htm
• http://jfmueller.faculty.noctrl.edu/toolbox/workshoptask.ht
m
• http://www.uwstout.edu/soe/profdev/rubrics.cfm
• http://www.learndash.com/formative-vssummative-assessment/
• http://www.introductiontorubrics.com/sampl
es.html