Outcomes Assessment Committee
2016-17
Monday November 7, 2016 (4 pm)
LDC 218
Goals for today
1. Introductions and welcome new members:
Mary Lou Papich & Leah Thompson
2. Review 2015-16 accomplishments
3. Review 2016-17 goals based on assessment plan
4. Organize into subcommittees
5. Time to meet in subcommittees
2015-16
accomplishments
Core learning ability (CLA) accomplishments
Assigned…
CLAs to ABE, ELL, ESLA, HSC and pre-college math and English
courses.
Assessed…
information literacy (IL), critical thinking (CT) and global
consciousness (GC) in PIP/DIP disciplines/programs that had IL, CT
or GC mapped.
Offered…
GC workshops during opening week (Sept) and on professional
development day (February).
Posted…
global consciousness posters in classrooms and student meeting
spaces.
Reviewed…
IL and CT data and presented to the OAC.
Selected…
quantitative literacy for 2016-17; refined definition and selected
rubric.
44%...
of submitted syllabi in spring 2016 had correct CLAs listed.
Program outcomes accomplishments in 2015-16
Assessed…
formally one program outcome for 12/14 prof-tech programs.
Refined…
program outcomes, curriculum maps, and outcomes plans.
25%...
of 131 prof-tech syllabi reviewed had program outcomes listed on
syllabi (spring 2016).
Course outcomes accomplishments in 2015-16
100%
of courses offered in spring 2016 had submitted course outcomes
to curriculum committee.
83%
of 359 course syllabi reviewed in spring 2016 had approved
course outcomes listed.
91% (61/66)
of full-time faculty and 51% (25/49) of adjunct faculty who were
required to submit a professional plan and report submitted a
course outcomes report.
Workshops...
Change that matters faculty education workshop
(2 adjunct and 13 full-time faculty participants)
Course outcome report mini-workshops
(8 adjunct faculty participants and 6 full-time faculty)
Student services learning outcomes accomplishments
Met…
individually with many student service directors.
Developed…
an SSLO plan and asked student service staff to assess one SSLO in
their SIP report in 2016-17.
Trained…
directors at one of their meetings.
NWCCU rubric for evaluating outcomes assessment plan
and progress
Criterion
Assessment
planning
Initial
No formal assessment
plan
Emerging
Relies on short-term planning
Developed
Clear multi-year plan
Highly developed
Clear multi-year plan with several
years of implementation
2
Assessable
outcomes
Non-specific outcomes.
Do not state student
learning outcomes.
Most outcomes indicate how
students demonstrate learning
Each outcome describes
student demonstration of
learning
Outcomes describe demonstration of
student learning. Outcomes used for
improvement.
3
Assessment Not clear that assessment Evidence collected. Faculty have
implementat data is collected
discussed relevant criteria for
ion
reviewing.
Evidence is collected and
faculty use relevant criteria
Evidence collected, criteria
determined and faculty discuss
multiple sets of data. Data is used
4
Alignment
5
Valid results
6
Reliable
results
7
Annual
feedback on
assessment
efforts
Results are
used
1
8
9
No clear relationship
between outcomes and
curriculum
Little to no evidence that
measures are valid
No process to check for
inter-rater reliability
Some alignment between
curriculum and outcomes
Clear alignment between
curriculum and outcomes
Curriculum, grading and support
services are aligned with outcomes
Majority of measures are valid
Valid measures in regular
use
Faculty check for inter-rater
reliability
Multi-year use of valid measures
No person or committee
provides feedback to
departments on quality of
their assessment plan
Results for outcomes are
collected but not
discussed
Occasional feedback by person or
committee
Annual feedback by person
or committee. Departments
use feedback
Results collected, discussed but
not used
Results collected, discussed
and used
Results collected, discussed, used
and evidence to confirm that
changes lead to improved learning
Attempts at aligning outcomes
and planning and budget
Alignment of outcomes and
planning and budget occurs
informally
Alignment of outcomes and planning
is systematic and intentional
Planning and Outcomes not integrated
budgeting
into planning and budget
Faculty preparing inter-rater
reliability
Multi-year use of process and
evidence of good inter-rater
reliability
Annual feedback, departmental use
and clear institutional support
2016-17 goals
(assessment plan )
CLA goals for 2016-17
Refine…
the definition and create a rubric for the 2017-18 CLA,
communication.
Explore…
curriculum maps for AST and ALS degrees and discuss with
learning contracts, co-op and honors courses.
List…
CLAs on at least 80% of syllabi.
Post…
quantitative literacy CLA posters in classrooms and student meeting
spaces.
Assess…
IL, CT, GC, and QL (PIP/DIP cycle).
Review…
IL, CT, and GC assessment data from previous year and determine
next steps.
Offer…
IL, CT, GC, and QL education based on assessment data.
Program outcomes goals for 2016-17
80%
of syllabi will have program outcomes listed.
100%
of programs will submit a program outcomes report assessing one
program outcome.
100%
of programs will refine their program outcomes, curriculum maps,
program outcomes plans.
Course outcomes goals for 2016-17
100%
of courses offered in winter & spring 2017 will have approved
course outcomes.
100%
of all full-time and adjunct faculty who are required to submit a
professional plan and report will submit a course outcome report.
95%
of syllabi of courses offered in spring 2017 will have the correct
course outcomes listed.
Conduct
Change that matters workshop.
Course outcome report mini-workshops.
Using small groups and teams to enhance student engagement
(Colleen McGoff)
Using game creation for concept synthesis (Colleen McGoff)
Tricks, tips and techniques for creating a community of learners in
your classroom (Darcie Donegan and others)
2016-17 work plans
1. Tresha and Anne Marie
2. OAC member workgroups
Tresha’s and AM’s 2016-17 work plan
Assist…
those in PIP/DIP cohort with IL, CT, GC or QL report.
prof-tech coordinators with program outcome report.
services with new student services learning outcomes plan and SIP report.
Coordinate…
with curriculum committee (e.g., new courses, curriculum maps).
with honors program, international courses, co-op and learning contracts on
course outcomes.
with faculty and chairs about missing syllabi and missing outcomes on syllabi.
Review
prof-tech CLA maps and program outcome issues in CurricUNET and course
requirements.
CLA assessment process
Develop…
reports through CurricUNET.
the definition and create a rubric for the 2017-18 CLA, communication.
Draft…
curriculum maps for AST and ALS degrees and assign CLAs to learning
contracts, co-op and honors courses.
Conduct…
Change that matters FEWs.
Train…
and assist faculty submitting a course outcomes report (problem of closing the
loop; will encourage faculty to submit a COR for the same CO 2 years in a row)
CLA assessment process
Current process
Potential ideas for new model
Embedded in 5 year PIP/DIP cycle
National normed assessment
One section of each course (every
5 years)
Adapt current process to be annual with all programs and
disciplines assessing one CLA annually
Adapt current process to be all sections every 5 years
Have an assessment day when all faculty are norming and
assessing student work (maybe a 2nd day to interpret
results)
Other ideas….
Kristina’s 2016-17 work plan
Collect and
track…
syllabi for inclusion of course outcomes, program outcomes, CLAs and
other items on syllabus checklist.
OAC members work groups
Work groups
Responsibilities
1. CLA analysis team
(Gretchen and Hilary)
Aggregate, analyze, and review 2015-16 CLA data. Respond to
CLA reports. Consult with TEAM about new assessment
process.
2. Information literacy
education (Margaret and
Mary Lou)
Review aggregated IL data. Research active TLA strategies.
Coordinate one IL mini-workshop.
3. Critical thinking education
(Lee and Leah)
Review aggregated CT data. Research active TLA strategies.
Coordinate one CT mini-workshop.
4. Global consciousness
education (Ines and new
student life director)
Review aggregated GC data. Research active TLA strategies.
Coordinate one GC mini-workshop with faculty, student
services, student life and intercultural center.
Research active TLA strategies. Coordinate QL session on
5. Quantitative literacy
education (John and Jason) professional development day.
Time to meet in subcommittees
Questions?
OAC members work groups
1. CLA analysis team (Gretchen and Hilary): Aggregate, analyze, and review
2015-16 CLA data. Respond to CLA reports. Consult with TEAM about
new assessment process.
2. Information literacy education (Margaret and Leah): Review aggregated
IL data. Research active TLA strategies. Coordinate one IL mini-workshop.
3. Critical thinking education (Lee and Mary Lou): Review aggregated CT
data. Research active TLA strategies. Coordinate one CT mini-workshop.
4. Global consciousness (Angela and new student life director): Review
aggregated GC data. Research active TLA strategies. Coordinate one GC
mini-workshop with faculty, student services, student life and
intercultural center.
5. Quantitative literacy education (John and Jason): Research active TLA
strategies. Coordinate QL session on professional development day.
CLA process – Initial goals (7 years ago)
1. To reduce the number of CLAs.
2. To ensure all students are exposed to and have a
chance to develop all 5 CLAs.
3. To minimize the burden on faculty.
4. To embed the process within existing structures.
5. To take a first stab at implementing the process and
using the data (i.e., rolled out one CLA/year).
6. To encourage discipline-level conversations.
7. To be able to use the results to determine which CLAs
students struggle with and excel at most.
8. To minimize the financial burden on the college (e.g.,
create a home-grown system).
CLA process – Challenges
1. Each discipline/program is assessing the CLAs only
once every 5 years (based on the PIP/DIP cycle).
2. The data tell us how students are doing at a point in
time – they don’t measure student growth.
3. The fact the CLAs are mapped to courses based on
designators may limit faculty attention to other CLAs.
4. CLAs are mapped based on designators, which ensures
all CLAs are addressed during the degree, but many
faculty believe the CLA mapped to their course is not
the primary ability improved through the course.
5. Too little data is available each year.
6. The quality of the data is compromised because of
individual faculty interpretation.
CLA process – Current goals
1. To be able to (1) measure students’ growth in
developing the CLAs from entrance to graduation and
(2) determine which CLAs students struggle with and
excel at most.
2. To ensure all students are exposed to and have a
chance to develop all 5 CLAs.
3. To have quality data – and nationally comparable data.
4. To allow faculty to focus conversations on assessment
results and on teaching, learning and assessment
strategies.
5. To minimize the burden on faculty.
6. To embed the process within existing structures.
CLA process – Next steps
1. Ed let cabinet know we are having these
conversations.
2. Tresha and AM are sharing this conversation with the
OAC.
3. AM will research national-normed options (e.g., CLA+,
ETS proficiency profile, ACT’s collegiate assessment of
academic proficiency, etc.).
The outcomes assessment committee
…is charged with promoting excellence in teaching and
learning at the course, program and college levels.
Specifically, the OAC
•
coordinates and monitors the college, program and
course outcomes assessment work as outlined in the
College’s assessment plan.
•
provides teaching, learning and assessment resources
to faculty and staff
•
provides educational forums to discuss teaching,
learning and assessment