EOF Directors’ Professional Development Seminar
“Program Planning & Assessment”
June 13, 2013
Rider University
Creating and Using Rubrics to Assess
Program Level Student Learning Outcomes
Mildred A. Mihlon, Ph.D.
Saint Peter’s University
Associate Vice President, Academic Affairs & Assessment
Who, What, Where &
Level of Familiarity with
Assessment?
Introductions
Effective program evaluation
must be grounded in a solid assessment
plan that is inclusive of a clear
vision/mission, relevant program goals
and well-articulated student learning
outcomes that can be evaluated
through direct and indirect measures
against defined standards of success.
Assumptions
Student Learning Outcomes…
Derived from the program mission and
goals, student learning outcomes are
identified actions that a student is
expected to demonstrate in terms of
knowledge, skills, and attitudes upon
completion of a program, course,
activity or process.
Student learning outcomes should be
measurable, manageable, and
meaningful.
Writing Student Learning Outcomes
Ask Yourself…
What are the most important things a student
should know, be able to do or demonstrate
after completing my program or from utilizing
my office/services?
• What does the end result look like? How will
you know the students have learned what you
want them to learn? How will you identify it?
What will they be able to do, say, think, care
about, or value after this experience.
•
Writing Student Learning Outcomes
Transform these attributes into
measureable, action-oriented
student learning outcome statements.
Examples:
Students will…
 Identify critical campus resources required for a successful
transition to college life.
 Demonstrate engagement through participation in campus life
activities.
 Develop effective self-management skills.
 Employ the use of appropriate decision-making skills.
 Apply the appropriate use of study skills and time management
strategies.
Writing Student Learning Outcomes
Mission
Goal 1
SLO 1
SLO 2
Summer Bridge Programs
Peer Mentorships
Transition Courses
Academic Advising
Goal 2
SLO 3
SLO 4
Learning Communities
Service Activities
Workshops
Orientations
• Direct Measure
SLO 1 • Indirect Measure
• Direct Measure
SLO 2 • Indirect Measure
• Direct Measure
SLO 3 • Indirect Measure
• Direct Measure
SLO 3 • Indirect Measure
Indirect Measures = imply learning has
occurred, but are not specific as to what
has been learned; often subjective.
Direct Measures = tangible evidence that
learning has occurred and to what extent
through observed changes in skills,
attitudes, values or knowledge base.
Middle States requires use of both!
Measuring Student Learning Outcomes
Direct Measures
Indirect Measures
Assignments/Exams
Grades/GPAs
Papers/Essays
Persistence/Graduation Rates
Projects
Surveys (NSSE)
Portfolios
Attendance
Student Reflections/Discussions
Job/Internship Placements
Employers Ratings
Interviews/Focus Groups
Capstone projects
Scores on GREs/LSAT/etc…
Exhibits
Honors/Awards
Presentations
Participation Hours
Measuring Student Learning Outcomes
Objectives:
 Define Rubric
 Rubric Features
 Construct a Rubric
 Aggregate Rubric Data
Creating & Using Rubrics

A rubric is a scoring instrument for evaluating
performance along a task-specific set of
criteria rather than a single numeric score.

Purposes of rubrics:
◦ Describe what “outstanding” or “satisfactory” or
“inadequate” performance looks like
◦ Articulate the criteria against which student
learning is judged
◦ Provide an objective guide for assessment
Definition & Rubric Purpose
A Rubric: Basic Features…
• Stated Objective or Purpose - title
• Scoring Criteria – characteristics of good
performance on the task
• Levels of Performance – defined degrees
of competency
• Descriptors –briefly define what is
expected at each level of performance
Basic Rubric Features
Presentation Rubric

http://www.rcampus.com/rubricshowc.cfm?code=Z79XC8&sp=true&
Peer Evaluation: Group Work

http://www.rcampus.com/rubricshowc.cfm?code=K78BBC&sp=true&
Self Evaluation for Group Participation

http://www.rcampus.com/rubricshowc.cfm?code=B78BAA&sp=true&
Individual Portfolio

http://www.rcampus.com/rubricshowc.cfm?code=K7369C&sp=yes&
Blog

http://www.rcampus.com/rubricshowc.cfm?code=N44XXC&sp=yes&
Student Participation
 http://www.siue.edu/~deder/partrub.html
Team Work
 http://course1.winona.edu/shatfield/air/team%20work.pdf
Journal
 http://course1.winona.edu/shatfield/air/reflectionrubric-1.pdf
Identify and define the assessment
objective or purpose (title)
2. Select and write the needed number of
scoring criteria (key attributes)
3. Select and write the desired levels of
performance
4. Select and write the descriptors
(features of scoring criteria)
1.
Steps to Creating A Rubric
1.
Identify and define the assessment
objective or purpose (title); statements
which identify the specific knowledge,
skill, or attitude the learner should gain
and display as a result of the
instructional activity.
Steps to Creating A Rubric
2. Select and write the needed number of
scoring criteria (key attributes); broad
concepts or specific tasks the student should
demonstrate when performing the activity.
• Examples of dimensions for a group exercise
analyzing a case study may include:
• Contribute to the group discussion
• Take responsibility for required work
• Value others viewpoints
• Analyze the study cooperatively
• Present the outcome in a given format
Steps to Creating A Rubric
3. Select and write the desired levels of
performance; descriptive levels of quality
starting with the worst quality up to the
best quality.
--Example of a 3 level gradation: poor,
average, excellent
--Example of a 4 level gradation: beginning,
developing, accomplished, exemplary
--Example of a 5 level gradation: poor, fair,
average, very good, excellent
Steps to Creating A Rubric
4. Select and write the descriptors
(features of scoring criteria); Start with
the best quality of each dimension. Simply
list the specific expectations you have for
the student. Then, for each level below
the best quality, identify the flaws or
missing elements which will cause the
student to lose points off the best quality
performance.
Steps to Creating A Rubric
Students are asked to write
a 5-page reflection paper
documenting their level of
college readiness following
participation in summer
transition program.
Creating A Rubric
Beginning 1
Developing 2
Writing
Quality &
Organization
Engagement
Study
Habits
Resource
Knowledge
Self-Efficacy
Creating A Rubric
Accomplished 3
Exemplary 4
Group Activity:
Develop a rubric to be used to
evaluate students’
leadership skills at the
conclusion of a peer
mentorship program.
Creating A Rubric
N = 75
Beginning 1
Developing 2
Accomplished 3
Writing
Quality &
Organization
15 (20%)
25 (34%)
30 (40%)
5 (6%)
Engagement
10 (13%)
15 (20%)
35 (47%)
15 (20%)
Study
Habits
15 (20%)
15 (20%)
40 (29%)
5 (6%)
Resource
Knowledge
5 (6%)
5 (6%)
60 (57%)
5 (6%)
Self-Efficacy
5 (6%)
10 (13%)
40 (54%)
20 (27%)
Aggregate Rubric Data
Exemplary 4
CONTACT
Mildred A. Mihlon, Ph.D.
Associate Vice President, Academic Affairs & Assessment
[email protected]