Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Greg Lampe – Associate Vice Chancellor
&
Gregg Nettesheim – Senior Information Manager
Office of Academic Affairs
February 2006 Conference on the First-Year Experience
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Challenging Context
The challenge: 13 campuses + a central office = 1 university
¾ Strong campus and department cultures
¾ A predisposition against assessment
Response:
¾ Commonly developed curriculum and policies
¾ Common assessment methods
February 2006 Conference on the First Year Experience
-1-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
ESFY Assessment Planning Process
¾ Intentional approach:
o Linking ESFY assessment to institutional mission and
priorities
o Reflecting critically on measures that could be useful
in decision-making and curriculum planning
¾ Integrated approach:
o Integrating standardized assessment instruments with
existing institutional measures
o Blending quantitative measures with qualitative
measures
¾ Focused approach:
o Limiting assessment strategies to key goals of
initiative
o Focusing on desirable outcomes of initiative
o Focusing on data that could be useful to multiple
constituencies
¾ Engaging approach:
o Engaging faculty and staff in the development of
assessment measures and tools
o Developing clear and coherent reporting methods of
key assessment findings
February 2006 Conference on the First Year Experience
-2-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Challenging Context
Residence Halls
The challenge: non-residential, commuter campuses
Difficulty fostering awareness of the program
¾
¾
Reduced sense of community
¾
Reduced participation in co-curricular activities
Response:
Use of in-class surveys whenever possible
¾
¾
Use of short, targeted web surveys
February 2006 Conference on the First Year Experience
-3-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
s
Se
ni
or
an
d
Ju
ni
or
s
Sp
ec
or
es
om
So
ph
Fr
es
hm
ia
ls
80
60
40
20
0
en
Percent
Challenging Context
The challenge: A two year university teaching a
freshman/sophomore baccalaureate, general
education curriculum
nd
¾ Reduced opportunity for follow-up in 2
year
¾
No follow-up possible within the Colleges in the 3rd
and 4th year
Response:
¾
Intensive monitoring in first year
¾
Emphasis on retention
¾
Follow-up with transfers within the UWS
February 2006 Conference on the First Year Experience
-4-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Meaningful Assessment
To be meaningful, assessment must be informative:
How does the First Year Experience support
achievement of the institutional mission?
How does the First Year Experience improve
student success and retention?
What does the First Year Experience tell us about
the overall effectiveness of the UW Colleges?
¾
¾
¾
To be meaningful, assessment must be useful:
Measure the effectiveness of the Engaging
Students in the First Year (ESFY) initiative
Inform changes to the ESFY initiative
Inform changes to curriculum and instruction
delivered to first year students
¾
¾
¾
February 2006 Conference on the First Year Experience
-5-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
University of Wisconsin Colleges Mission Statement:
The University of Wisconsin Colleges is a multi-campus
institution that prepares students for success at the baccalaureate
level of education, provides the first two years of a liberal arts
general education that is accessible and affordable, and advances
the Wisconsin Idea by bringing the resources of the University
to the people of the state and the communities that provide and
support its campuses.
Engaging Students in the First Year Mission Statement:
The University of Wisconsin Colleges’ first year of college
experience will focus on first year students as they make the
transition from their high school or post high school experiences
to the college experience. This intentional, comprehensive
curricular and co-curricular initiative will promote an
understanding and appreciation of liberal education, will assist
students with developing strategies and attitudes that will
maximize academic success, familiarize students with campus
resources and how to use them, foster the development of
positive relationships between and among students, faculty, and
staff, and prepare students to become life-long learners,
responsible citizens and leaders.
February 2006 Conference on the First Year Experience
-6-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Coherence and Integration
Conduct assessment that is consistent with:
¾ The mission of the UW Colleges
o The UW University of Access
o Freshman/Sophomore transfer institution
¾ The mission and goals of the ESFY program
o Assess the various curricular, campus
community and student success aspects of
ESFY.
¾ The UW Colleges student populations
o Level of readiness for college
o Curricular needs
o Barriers to success
o Academic aspirations
February 2006 Conference on the First Year Experience
-7-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Coherence and Integration
Integrate assessment of the first year experience with:
¾ Other assessments being conducted
o Assessment of general education
o Assessment of student engagement
¾ Analysis of student retention
¾ Reporting of university outcomes
February 2006 Conference on the First Year Experience
-8-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Methods and Instruments
Survey Instruments:
¾ First-Year Initiative from EBI Inc.
o Designed specifically for first year students
o Good fit for the UWC ESFY initiative
¾ Community College Survey of Student
Engagement (CCSSE)
o Learning centered indicators
o Broadly administered
¾ Internal Surveys
o Annual survey of student goals and intentions
o Orientation survey to new freshmen
o Foundations of Excellence self-study survey of
students and faculty.
February 2006 Conference on the First Year Experience
-9-
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Methods and Instruments
Assessment of general education proficiency
Proficiencies:
a. Analytical Skills
b. Quantitative Skills
c. Communication Skills
d. Aesthetic Skills
b. Quantitative Skills Performance
Indicators
1. Solve quantitative and mathematical
problems
2. Interpret graphs, tables and
diagrams
3. Use statistics appropriately and
Assessed Students by Prior Credits
Summarizes Assessment Performance
by Course Based on Prior Completed
Credits
Geography&Geology
Freshmen
Sophomores
Counts
Prior Credits →
0
1-29
30+ total
13
14
8
35
GEO Exceeds Expectations
101
Meets Expectations
20
22
6
48
Fails to Meet Expectations
11
5
8
24
B2 – Interpret graphs, tables and diagrams
February 2006 Conference on the First Year Experience
- 10 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Methods and Instruments
Incorporate First Year Assessment into other Assessments
Retention to a 3rd Term by High School Rank
and FYS Enrollment
6 Year Average
Percent Retained
80%
60%
40%
20%
0%
No FYS FYS - Bottom No FYS - Top
Bottom Half
Half
Half
FYS - Top
Half
Percent Retained
Retention to a 3rd Term by Curricular Group and
FYS Enrollment
2 Year Average
80%
60%
40%
20%
0%
Group A Group A Group B - No Group B - No
Mat110 and
Mat110 and Degree Math or Degree Math or
Eng102 and no Eng102 with English and no English with
FYS
FYS
FYS
FYS
February 2006 Conference on the First Year Experience
- 11 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Significant Findings and Lessons Learned
In the First-Year Seminar:
¾ Reading materials matter
¾ Students value survival skills over academic skills
¾ Enrollments in FYS are too conventional
o Predominantly traditional age students
o Predominantly full-time students
In the first year students are seeking more interaction:
¾ With other students
¾ With instructors
February 2006 Conference on the First Year Experience
- 12 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Significant Findings and Lessons Learned
Enrollment in First-Year Seminars improves retention,
but the benefits may not be uniformly distributed
Percent Retained
Retention to a 3rd Term by Curricular Group
and FYS Enrollment
2 Year Average
100%
80%
60%
40%
20%
0%
Group A Mat110
and
Eng102
and no
FYS
Group A Mat110
and
Eng102
with FYS
Group B No
Degree
Math or
English
and no
FYS
Group B - Group C Group C
- Dev.
- Dev.
No
Degree Math and Math and
Dev.
Dev.
Math or
English English English
with FYS and no with FYS
FYS
February 2006 Conference on the First Year Experience
- 13 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Significant Findings and Lessons Learned
¾ First year students are students in transition
¾ Acquiring or relearning general education skills
¾ Need and want higher education survival skills
¾ Looking for academic direction
¾ Looking for direction beyond academia
February 2006 Conference on the First Year Experience
- 14 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Significant Findings and Lessons Learned
¾Distribute ownership of the initiative
¾Beware of over-assessment
¾Integrate first-year assessment into
existing assessment practices, and vice
versa
¾Maintain momentum
February 2006 Conference on the First Year Experience
- 15 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Appendix A – Environmental Scan
Campus Participation in ESFY Activities
Activity
First-year seminars
Intro. To college programs
Common reading
Skills workshops/guest speakers
Fall campus convocation
Activity “passport” model
Study abroad
New student special events
Career center
Enhanced curricular offerings
(learning communities, service
learning, project-based learning, etc.)
2003
12
12
7
9
5
5
7
8
6
2004
12
13
5
11
7
9
6
7
11
2005
12
13
8
13
10
10
9
11
9
Na
11
12
February 2006 Conference on the First Year Experience
- 16 -
Assessing the First Year of College Experience:
A Multifaceted, Integrated Approach
Questions, Comments and Suggestions
Greg Lampe – [email protected]
Gregg Nettesheim – [email protected]