PROGRAM
ASSESSMENT:
The
Administrator’s Role
Chrissy L. Davis Jones
Administrators’ Institute
Spokane Falls Community College
February 2015
© 2015 NACADA The Global Community for Academic Advising
NACADA Executive Office at Kansas State University
2323 Anderson Ave, Suite 225, Manhattan, KS 66502-2912
P: 785-532-5717 | F: 785-532-7732 | E: [email protected]
The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless
otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government
officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be
reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written
permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own
use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or
subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except
with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal
law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of
the National Academic Advising Association.
OUTCOMES
Participants will…
• learn basic assessment terminology;
• identify two (different) tools they can
use to assess the advising program
on their campuses;
• be active learners during this session.
FOUNDATION: TERMINOLOGY
• Evaluation: a process of examining or reviewing individuals or programs to
measure performance.
• Evidence: outcomes that establish support of behaviors, attitudes, or (external)
attributes.
• Lagging indicators: Measure goal accomplishment (easy to measure but hard
to influence)
Leading indicators: Predict goal achievement (maybe more difficult to
measure)
Mapping: process to determine when, where, and how outcomes for advising
will be attained
• Multiple measures: several measure of the same construct
• Process outcome: expectations about the process of delivery of academic
advising across the institution; focus on service vs. advisors
• Programmatic objective: statements of what the program wants students
and/or advisors to be able to do and know (tend to be more specific than goals)
• Student Learner Outcome (SLO): statements of what students can expect to
learn (i.e., knowledge, skills, and/or value) from the advising experience
ASSESSMENT
Assessment is an intentional,
systematic research-based process of
collecting, interpreting, and utilizing
data from multiple sources and
techniques (Upcraft & Schuh, 1996;
Robbins, 2009) to focus on enhancing
student learning and development
(Palomba & Banta, 1999; McClenney &
McClenney, 2004)
PURPOSES
of
Assessment
“INSTITUTIONAL CURIOSITY”, I.E. STUDENT LEARNING AND STUDENT
DEVELOPMENT (MAKI, 2002, 2004).
INTENTIONS of Assessment
• Assessment is intended to be a positive
process, yet, its connotations are often
negative (Nutt, 2013)
• The focus has often been on activities that
demonstrate accountability to the exclusion
of those that are aimed at improvement
(Nutt, 2013).
• Our perspectives often influence
receptiveness
EVALUATION
Evaluation is part of the
assessment process, in which
the results are analyzed and
transformed into a format that
is useable (Upcraft & Schuh,
1996; Upcraft, 2003).
ELEMENTS
of program
assessment for
Advising Administrators
ASPIRE
•VISION
PURPOSE
•MISSION
GENERIC
ACTIONS
•GOALS
SPECIFIC
ACTIONS
•OBJECTIVES
END
RESULTS
•OUTCOMES
Mission Statement
CHECKLIST
What is the purpose of academic advising
on your campus?
What does your institution and advising
unit(s) value about academic advising?
How do these elements tie into the overall
assessment plan for academic advising?
In order to provide high-quality learning opportunities,
the mission of academic consulting is to support
students in achieving their goals through building an
effective working relationship between student and
consultant.
An effective academic consulting working relationship
provides a holistic approach: guiding students to
relevant resources to foster their success;
collaborating with others who have complementary
expertise; and designing an educational, engagement,
and academic success plan based on the students'
values, attributes, learning styles, and
academic/career/goals.
© 2015 NACADA The Global Community for Academic Advising
NACADA Executive Office at Kansas State University
2323 Anderson Ave, Suite 225, Manhattan, KS 66502-2912
P: 785-532-5717 | F: 785-532-7732 | E: [email protected]
The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless
otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government
officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be
reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written
permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own
use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or
subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except
with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal
law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of
the National Academic Advising Association.
DEVELOPING
Goals & Objectives
Both goals and objectives MUST:
support the mission statement
&
serve as a guide for learning
outcomes
Goal
The purpose toward which an endeavor is
directed.
Something that one's efforts or actions are
intended to attain or accomplish; purpose;
target.
I want to achieve success in the field of
genetic research and do what no one has
ever done.
I want to complete this thesis on genetic
research by the end of this month.
Generic action, or better still, an outcome
towards which we strive.
Specific action - the objective supports
attainment of the associated goal.
Goals may not be strictly measurable or
tangible.
Must be measurable and tangible.
Longer term
Mid to short term
Meaning
Example
Action
Measure
Time frame
Objective
Learning
OUTCOMES
Learning outcomes are what we expect
student to know, do, value, and
appreciation as a result of their
academic advising experience.
Mapping: The process of
determining when, where,
and how the advising
outcomes will be
accomplished over the
duration of students’
academic journeys.
Mapping
Exercise
(see handout)
After mapping out the learning
experience using the mission
statement, common goal, and
objectives…
What’s next?
Needs ASSESSMENT
Tools for constructive and positive
change that is rational, logical, and
functional transformation---not
change driven by controversy, quick
fixes, and situational crises.
Sample Questions
• How efficient is the program?
• How is effective practices measured?
• What is the overall satisfaction rating?
• Are student learning outcomes being met?
• Are advisors trained at the appropriate level of
experience needed?
• What plan is in place to ensure continuous
professional development for advisors?
DATA Collection
• Attitudinal & Learning surveys/questionnaires
• Student focus groups
• Student forums
• Existing data approach
• Key Informant
*Know how you will respond to the data before
it arrives
*No instrument is perfect, so plan for multiple
measures
NOTE!
At least one of your
measures should
include existing
institutional data
CAUTION!
Satisfaction Surveys
Lagging & Leading
INDICATORS
LAGGING INDICATORS
Lagging indicators---measure goal accomplishment
*Easy to measure but hard to influence
*Tells you little to nothing about how the outcomes will be
achieved
*There is little early warning(s) about being on track
to achieve the goal(s).
LEADING INDICATORS
DATA TO GUIDE FUTURE ACTIONS
Leading indicators---Predict goal
achievement
* Indicators can be influenced, but the
indicators may be more difficult to measure
* Tells you how the outcomes are achieved
* There are early warning(s) about being on
track to achieve the goal(s).
LAGGING & LEADING INDICATORS EXAMPLE
© 2015 NACADA The Global Community for Academic Advising
NACADA Executive Office at Kansas State University
2323 Anderson Ave, Suite 225, Manhattan, KS 66502-2912
P: 785-532-5717 | F: 785-532-7732 | E: [email protected]
The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless
otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government
officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be
reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written
permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own
use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or
subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except
with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal
law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of
the National Academic Advising Association.
DEVELOPING LEADING INDICATORS
• What are your institution’s goals?
• What processes support the achievement of
the goals?
• What are your institution's SLOs?
• What individual actions comprise the
processes?
• Which actions can be affected and will make an
improvement?
– What data do you need to know?
– When and how often do you need it?
CONTINUE…
Mapping
Exercise
(refer to handout)
SFCC’S
ADVISING SLO CONTINUE…
SLO #2: Students will know where and how to
access appropriate academic support services to
meet their academic goals.
Questions to consider:
•What would be the model path to achieve this goal?
• What individual actions comprise this process?
• Which actions can be affected and will make an
improvement?
• What do you need to know (i.e. data) and when to
affect the outcome?
OVERVIEW OF
THE PROCESS
Sharing the Results
Interpreting the Results
Follow-Up & Follow Thru
Implications for Future Practices
Culture of Evidence
TO CONCLUDE
Assessment of academic advising is about…
• developing (realistic) expectations for learning;
• gathering evidence to understand students’ academic
journeys;
• using evidence to support improvement of the
advising system that will contribute to continuous
quality improvement of the learning process and
student achievement.
• Talk to and work with the experts---IR staff
THANK YOU!