Writing Services Learning Outcomes: No
‘Understanding,’ ‘Appreciating,’
or ‘Learning’ Allowed!
Workshop for services
Tresha Dutton
Anne Marie Karlberg
March 18 (11 am) or April 26 (2 pm) or May 10 (10 am), 2013
Outcomes: Employees will be able to…
1. explain the purpose of writing clearly
articulated learning outcomes.
2. distinguish between effective and ineffective
outcome statements.
3. write a learning outcome for their service.
Review from SIP workshop: Goals
Is the goal a direct
indicator, indirect
indicator, or
institutional data?
Direct indicators
(employees/faculty/
student learning
outcomes)
Select up to 3 strategic plan
objectives to which each goal
is connected (use the dropGoals for 2013-14
down list(s) below)
1. Employees will
5.4. Foster an evidence-based
be able to use
culture of continuous
data to inform
improvement
decision-making
2.3. Improve student learning
processes.
2.2. Improve instructional
delivery
2. Faculty will
Institutional data
5.4. Foster an evidence-based
revise course
(numbers, rates, etc.) culture of continuous
outcomes for
improvement
courses
2.3. Improve student learning
representing 50%
2.2. Improve instructional
of course
delivery
enrollments.
3. WCC employees
are satisfied with
the quality of
services
provided.
Indirect indicators
(from surveys, focus
groups, interviews)
5.4. Foster an evidence-based
culture of continuous
improvement
Resource
implications
Maintain level
of employees.
Maintain level
of employees.
Consider
software for
managing
curriculum and
outcomes
process.
Maintain level
of employees.
Review from SIP workshop:
Progress on goals
Goal 2: Faculty will revise course outcomes for courses
representing 50% of course enrollments.
Evaluators
Source(s) of
data
(Evidence)
Tresha Dutton and Anne Marie Karlberg
Course forms reviewed by curriculum committee
and the Excel spreadsheet titled “V:\Outcomes
Assessment\MASTER 2011-12-Course And CLAWorkPlan-041912.xlsx”.
Baseline Course “objectives” were identified on course
information summary forms. These course objectives had not
(2012-13) been updated for years and the outcomes were not
written in measureable terms.
Review from SIP workshop:
Progress on goals (continued)
Goal 2: Faculty will revise course outcomes for courses
representing 50% of course enrollments.
Summary of
assessment
evidence
(findings)
During the fall and winter quarters, the outcomes assessment
coordinator and the director of AIR facilitated several faculty
workshops – called “Writing Effective Course Outcomes: No
‘Understanding,’ ‘Appreciating,’ or ‘Learning’ Allowed!” –
about how to write effective course outcomes. Fifty-eight fulltime faculty and 12 adjunct faculty / prof-tech coordinators
attended these sessions.
During 2011-12, faculty articulated course outcomes for
courses representing 68% (31,509 / 46,364) of course
enrollments. 57% (26,595/46,364) were formally approved
through curriculum committee by June 30, 2012. 29%
(201/694) of all courses offered at WCC have submitted
course outcomes.
Review from SIP workshop:
Progress on goals (continued)
Goal 2: Faculty will revise course outcomes for courses
representing 50% of course enrollments.
Potential
causes for
significant
changes
Faculty were provided an increase in pay for participating in the
“Writing Effective Course Outcomes workshop”.
Curriculum committee and outcomes assessment committee
members joined forces to form sub-committees that processed the
high volume of submitted course forms.
Past use of NA
results for
improvement
Next steps for In 2012-13, the outcomes assessment coordinator and the director
improvement of AIR will conduct faculty workshops to train faculty to create
rubrics (with a goal of 1% of courses will have a rubric developed).
Faculty will revise outcomes for courses representing 75% of
WCC’s enrollment or 35% of all courses offered and will include
course outcomes on syllabi (75%).
Status Completed or met/exceeded expectations
Assessment information
Level
Type of data College
Program
Direct
indicators
(learning
outcomes)
Services
learning
outcome(s)
Indirect
indicators
(surveys)
Student or
employee
feedback on
Institutional
data
(numbers
and rates)
Number of
students or
employees
using service.
Course
Assessment information
Services
Level
Type of data College
Program
Direct
indicators
(learning
outcomes)
Services
learning
outcome(s)
Indirect
indicators
(surveys)
Student or
employee
feedback on
Institutional
data
(numbers
and rates)
Number of
students or
employees
using service.
Course
Assessment information
Level
Type of data College Program
Services
Direct
indicators
(learning
outcomes)
Services learning
outcome(s)
Indirect
indicators
(surveys)
Student or employee
feedback on learning
outcomes and
experience
Institutional
data
(numbers
and rates)
Number of students
or employees using
service.
Numbers of XXX
processed
Course
Direct indicators
Level
Require
students or
employees to
demonstrate
their learning
Type of data College Program
Services
Direct
indicators
(learning
outcomes)
Services learning
outcome(s)
Indirect
indicators
(surveys)
Student or employee
feedback on learning
outcomes and
experience
Institutional
data
(numbers
and rates)
Number of students
or employees using
service.
Numbers of XXX
processed
Course
Direct indicators: Examples of indicators
for a service’s learning outcome
Level
Type of data College Program
Services
Direct
indicators
(learning
outcomes)
Learning outcome:
Employees will be able to
conduct a hiring process.
Indicators:
•
employees submit
interview questions for
review prior to
interview
•
employees refrain from
putting any applicant
name information in
email communications
to one another
•
the committee chair
submits a strength and
weakness memo to HR
Course
Indirect indicators
Level
Ask students
or employees
to provide
feedback on
their learning
or
experiences
through
surveys,
focus groups,
interviews
(perceptions
of their
experiences)
Type of data College Program
Services
Direct
indicators
(learning
outcomes)
Services learning
outcome(s)
Indirect
indicators
Student or employee
feedback on learning
outcomes and
experiences
(perceptions)
Institutional
data
(numbers
and rates)
Number of students
or employees using
service.
Numbers of XXX
processed
Course
Indirect indicator: Example goal statement
Level
Type of data College Program
Services
Indirect
indicators
Employees will report
feeling confident about
hiring processes.
(perceptions)
Evidence
• internal surveys,
forms, interviews,
focus groups
• services surveys
(employee and
student feedback in
2012)
• student opinion
survey (2008)
• CCSSE (2010)
Course
Institutional data
Level
Reflect the
overall
condition
and
effectiveness
of the service
Type of data College Program
Services
Direct
indicators
(learning
outcomes)
Services learning
outcome(s)
Indirect
indicators
(surveys)
Student or employee
feedback on learning
outcomes and
experience
Institutional
data
(numbers
and rates)
Number of students
or employees using
service.
Numbers of
documents
processed per hour.
Course
Institutional data: Example goal statement
Level
Type of data College Program
Services
Institutional
data
(numbers
and rates)
HR will have a 100%
success rate for
hiring processes
during 2013-14.
Course
Activity #1
Connect each learning outcome or goal to
its respective service.
Value of clearly articulated goals
Guides service activities
Creates consistency of services activities
Provides framework for service assessments
• identifies targets/indicators for evaluation of service
• focuses attention on type(s) of data
• structures data to allow for interpretation of services
improvement(s)
Service goal: Learning outcomes
Level
Require
students /
employees to
demonstrate
their learning
Type of data College Program
Services
Direct
indicators
(learning
outcomes)
Services learning
outcome(s)
Indirect
indicators
(surveys)
Student or employee
feedback on learning
outcomes and
experience
Institutional
data
(numbers
and rates)
Number of students
or employees using
service.
Numbers of XXX
processed
Course
A learning outcome is…
a measurable statement that defines what
students, employees, or others will be able to
do by the end of an experience or interaction
(i.e., the skills).
Effective learning outcomes…
1. begin with the phrase, “Students/employees will be able to…”
(verbs from the cognitive, affective, or psychomotor domains).
2. are limited in number (i.e., 1-3 maximum).
3. use active verbs that are measureable, quantitatively or
qualitatively, and focus on what students/employees will be
able to do.
4. are stated clearly (i.e., be specific).
5. are stated concisely (e.g., eliminate filler words, be succinct).
6. separate distinct sets of skills.
7. bundle multiple related skills if they are steps in a process.
8. may specify criteria under which the behavior occurs.
9. may specify conditions under which the behavior occurs.
1. Begin with the phrase…
“Students/employees will be able to…”
Choose verbs from the cognitive, affective, or
psychomotor domains (see the “How to write services
learning outcomes” hand-out).
Domain
Examples
Cognitive
identify, differentiate, evaluate
Affective
accept responsibility for, express, co-operate
Psychomotor
design, build, operate
2. Are limited in number (i.e., 1-3)
Focus on the most important outcome(s) for
your service.
Agree on outcome(s) with service employees.
Be realistic.
3. Use active verbs that are measureable,
quantitatively or qualitatively
Focus on what students/employees will be
able to do (i.e., skills), rather than on what
they will learn or activities they will do.
3. Use active verbs that are measureable
Avoid…
appreciate
be aware of
be familiar with
be shown
be taught about
demonstrate
understanding
do, learn
know
practice
understand
Replace with…
critique
identify
explain
categorize
distinguish
review
determine
operate
manipulate
3. Use active verbs that are measureable
…examples
Not so good: Students will understand how to submit a
completed FAFSA.
Better: Students will be able to submit a completed FAFSA.
Not so good: Employees will know about their benefit
information.
Better: Employees will be able to access their benefit information
online.
Not so good: Employees will be taught about data to inform
decision-making processes.
Better: Employees will be able to use data to inform decisionmaking processes.
4. Are stated clearly (be specific)
Not so good: Students will be able to identify
different interesting careers.
Better: Students will be able to identify careers that
match their personality type and skills.
5. Are stated concisely (e.g., eliminate
filler words, be succinct etc.)
Not so good: Students who use judicial affairs
services will be able to articulate effectively how
their actions will impact their relationship with the
college, other students, and faculty and staff.
Better: Students who use judicial affairs services will
be able to articulate how their actions will impact
their relationship with students and employees.
Write the learning outcome at a competent level!
6. Separate distinct sets of skills
Not so good: Employees will be able to locate the
College’s safety plan and explain its emergency
procedures.
Better:
Employees will be able to locate the College’s
safety plan.
Employees will be able to explain the College’s
emergency procedures.
7. Bundle multiple related skills
Multiple related skills can be “bundled” together in
one statement if they are steps in a process.
Not so good:
Students will be able to complete a College
application for admission form.
Students will be able to submit a College
application for admission form.
Better: Students will be able to complete and
submit a college application for admission form.
8. May specify criteria under which the
behavior occurs
Criteria (standards): accuracy, quality, time constraint,
quantity
• “with no more than 1 incorrect entry”
• “according to FERPA regulations”
Examples:
Students will be able to submit a completed FAFSA
with no more than 2 errors.
9. May specify conditions under which the
behavior occurs
Conditions: given “X” or without “Y”
• “after accessing Fidalgo”
• “using the faculty handbook”
Examples:
After attending new employee orientation, faculty
and staff will be able to locate the payroll office.
Activity #2
Using the checklist, evaluate and correct the poorly written services
learning outcomes.
1. Faculty and staff will be able to understand the criteria for making
budget requests and will recognize the next steps in their budget
development process.
2. Students, faculty, and staff will be able to access information in a
timely manner through multiple channels and be informed about
college programs, services, news, events, and governance activities.
3. Students will be able to demonstrate an awareness of their
community, and the role and responsibility that they play within
the community.
4. Students will be taught about how to seek additional funding to
help pay college costs, find the scholarship offerings, identify
scholarships that match their academic qualifications, and
successfully complete a scholarship application for consideration.
SIP progress report:
Linking evidence to outcomes
What evidence will you collect and use to
determine the extent to which the learning
outcome(s) and goal(s) are being achieved?
Examples of indicators for a service
learning outcome
Level
Type of data
Direct
indicators
(learning
outcomes)
College
Program
Services
Learning outcome: Employees
will be able to conduct a hiring
process.
Indicators:
•
employees submit interview
questions for review prior to
interview
•
employees refrain from
putting any applicant name
information in email
communications to one
another
•
the committee chair submits a
strength and weakness memo
to HR
Course
Linking evidence to outcomes
Learning outcome: Employees will be able to conduct a
hiring process.
Level of Performance
Indicators
(description of exemplary
level of performance)
1. employees submit
interview questions for
review prior to
interview
2. employees refrain from
putting any applicant
name information in
email communications
to one another
3. the committee chair
submits a strength and
weakness memo to HR
Exemplary
Accomplished
Developing
Beginning
11
3
0
1
Total
15
4
8
2
1
15
2
2
3
8
15
Activity #3
Write a learning outcome for your service.
In pairs, analyze your outcomes using the
checklist.
AIR website
SIP resources
Next steps for learning outcomes
1. Collaborate with service employees.
2. Write one learning outcome, if
applicable, for your SIP report.
3. Submit your SIP report by May 31, 2013,
to your respective VP.
Questions?