A Dozen Ideas about
First-Year Assessment
FYE Annual Conference
Atlanta, GA
February 26, 2006
Bradley E. Cox
Coordinator of Research and Public Information, National Resource
Center for The First-Year Experience and Students in Transition
Randy L. Swing
Co-Director & Senior Scholar, Policy Center on the First Year of
College
Fellow, National Resource Center for The First-Year Experience and
Attendance & Early Intervention
 Does
it matter if students
miss class?
 Are all class cuts the
same?
 Do colleges “set up”
students to miss class?
A Study
of Attendance Patterns
University of Mississippi
Freshman Absence-Based Intervention (pilot)
Experimental design (treatment & control groups)
Students who missed 2 classes/8 weeks were
reported
Graduate student made personal contact
100
Cum GPA =
C or better
(end of term)
Anderson, C. (2004).
Percent
80
60
87%
40
55%
20
0
Intervention
None
Institutionalizing an
Intervention
(year 2)
University of Mississippi
Pilot taken campus-wide
Students who missed 2 classes/8 weeks were
reported
Residence Hall Advisor made personal contact
100
87%
55%
Cum GPA =
C or better
(end of
term)
Anderson, C. (2004).
Percent
80
70%
60
40
20
0
Pilot
None
Year 2
Attendance Patterns - Semester
weeks 1-4
weeks 5-8
weeks 9-12
weeks 13-16
Average number of class cuts per 2 week period
Anderson, C. (2004).
Grade Point Average
by Perfect or Not Perfect Attendance
4.00
3.04
2.67
3.00
3.15
2.76
3.00 2.88
3.09
2.82
2.00
1.00
NS
0.00
1
2
Perfect Attend.
3
4
Not Perfect
On average, students with perfect attendance
had higher grade point averages – statistically
significant in periods 1, 2, and 4.
Anderson, C. (2004).
GPA & Absence = Oversleeping
4
3
2.99
2.95
2.59
2.6
2.87 2.96
2.97
2.64
yes
2
no
NS
1
0
1
2
3
4
Difference in GPA is statistically significant in periods 1,
2, & 4. Not in period 3.
Anderson, C. (2004).
GPA & Absence = Too Tired to Go
3.5
3
2.5
2.97
2.94
2.39
2.87 2.91
2.56
2.73
2.94
2
yes
1.5
NS
1
NS
0.5
0
1
2
3
Significant in periods 1, & 2.
Not significant for period 3 & 4.
Anderson, C. (2004).
4
no
GPA & Absence = Minor Illness
4
2.95
3
2.77
2.91
2.83 2.91
2.95
2.64
2.29
yes
2
no
NS
NS
NS
1
0
1
2
3
4
Significant only for first period of term.
Anderson, C. (2004).
We can’t get there!
Porter and Umbach studied the time it takes
for students to move between classrooms.
Findings....
First-year students attend classes in more
buildings and travel the greatest distances
Some class changes were “impossible”
Advisors did not consider distance in making
schedules.
Porter, S. R., & Umbach, P. D.
Retention

Put it into context



Calculate the savings


Prediction formulas
Peer institutions
Financial Implications
Find out what happens next

Student persistence at other institutions
Predictors of College Graduation

Biggest predictors of graduation
1.
2.
3.
4.

HS GPA (30.84% of variability)
SAT (additional 4.02%)
Gender (additional 0.73%)
Race (additional 0.42%)
Additional input & environment
variables improve predictive value
(R2) to 51.5%.
Source: Astin & Oseguera, 2005
Retention Rate Comparison
Detailed information from all member schools. Allows
comparison of selected peers. Breakdown on race & gender.
Includes 1yr & 2yr retention. ?Includes graduation?
National
Average
Consortium for Student Retention Data Exchange &
ACT
Calculate the Savings
Noel-Levitz Retention Revenue Calculator
Find out where they go
Where do your transfer-outs go?
Where do your recruits go?
Where have your transfers come from?
www.studentclearinghouse.com
Before & After
How do students change in the first year?
Pre/post surveys
pretest at orientation
posttest at end of the first year
match on student id numbers
Pre/Post Survey Pairings
CSXQ-CSEQ
Identify
student
expectations at
orientation with
CSXQ.
Use
BCSS – NSSE
BCSS
currently in
large-scale pilot
testing.
NSSE
among most
popular current
instruments.
CSEQ at end
of first year (or
later) to compare
expected vs. actual Measures student
experiences.
engagement in the
learning process
Addresses student
time use and selfreported gains.
CIRP – YFCY
CIRP
Freshman
Survey has 40
years of
longitudinal data.
YFCY
has both
independent and
linked questions for
analysis of change
over first year of
college.
Expectations vs. Reality
Go to Art Gallery or Exhibit on Campus
80
75
70
73.7%
73.7
65
Reality = 36% fewer
students than
expectation.
60
55
50
45
38%
38
40
35
38.2
38.2
%
30
Expect
Expected (orientation)
Source: CSEQ National Data
Yr 1
End of 1st Year
Yr 2
End of 2nd Year
Change in First-Year Students’
Activities and Feelings
Self-Report: “Above Average” or “Highest 10%” in
Emotional Health
100%
-12.3%
80%
60%
40%
20%
0%
55.7
%
At Entry
Anticipated behavior:
43.4
%
End of First Year
Actual first-year
behavior:
“Some” or “good
chance” I will seek
personal counseling:
Sought personal
counseling:
30.1%
17.4%
Example from CIRP and YFCY data
Religion and Spirituality
“Frequently” or
“Occasionally” attend a
religious service
100%
80%
60%
40%
20%
At Entry
81%
Life goal: Integrating
spirituality into my life
is “very important” or
“essential”
100%
End of First
Year 49.6%
80%
60%
End of First
Year 55.7%
0%
25.3%
Example from CIRP and YFCY data
40%
20% At Entry
0% 40.5%
+
9.1
Student Voices

Ask them questions




KISS & QOTD/W
Prompt their thinking
Consider institutional “promises”
Let their pictures tell the story

Reflexive photography
KISS & QOTD/W





Keep it… Simple, Short, Straight, Strong
Asking questions & getting answers
Students talk to students
Administrators like short & straight to
point
Question of the Day/Week


Easy enough for one person office
Directly addressing timely issues
Cox, B.E. & Stirton, E. R (FYA-list essay)
Prompts Project


Quick, simple way to get lots of data
10 minute response to open-ended prompts
during required English 101 or FY seminar



Write for 10 minutes about your hopes, dreams
fears, and expectations for this semester.
What is partying like here at VCU? How do you
feel about the role of partying at VCU?
What types of serious crises have you and your
friends experienced this semester, and how
have you coped?
Claffey & Laff (2004)
Promise Audit

Consider the various “promise-makers”



Find out which promises are being broken.


Staff, websites, publications, etc.
Friends, media, the “grapevine”?
Can use existing collection processes
What are the effects of broken promises?
Claffey & Laff (2004)
Reflexive Photography

Pictures worth 1,000 words



Pictures plus words  Priceless
Students love to take pictures
Inexpensive and very powerful


Costs: Less than $100 plus staff time
Benefit: Images verify student
statements
Harrington, C.E. & Schibik, T.J. (2003)
Student Voices: Example

“The campus housing at USI is supposed to be
“dry” (alcohol free), but every weekend after
all the partying is over this is all that remains.
Sure, we have a “dry” campus!” - Mary
Harrington, C.E. & Schibik, T.J. (2003)
Student Voices: Example

“New construction project. This is a very
good sign that the campus is growing and
that more space is needed.” - Kirk
Harrington, C.E. & Schibik, T.J. (2003)
Student Voices: Example

“My roommates took this picture of me in the
morning one day. This picture means feeling real
tired in the morning after staying up late
studying. I was thinking how upset I was because
they were making noise outside doing
construction and it woke me up.” - Mercedes
Harrington, C.E. & Schibik, T.J. (2003)
Student Voices: Example

“This is the mail truck. It is amazing how personal
people can be when they write. I have become closer
to my friends through letters. I also feel like my mom
has become one of my best friends rather than just a
parent. The mail has taught me a lot about myself
and the important people in my life.” - Abby
Harrington, C.E. & Schibik, T.J. (2003)
(Baker’s) Dozen Ideas
1.
2.
3.
4.
5.
6.
Attendance
Intervention
Distance
Calculation
Prediction
Formulas
Peer Institutions
Financial Value of
Retention
Transfer-Out
Tracking
7.
8.
9.
10.
11.
12.
13.
CSXQ & CSEQ
BCSS & NSSE
CIRP & YFCY
Questions of the
Day/Week
Ten Minute
Prompts
Promise Audit
Reflexive
Photography
Additional (free, web) Resources

FYA-List



Typology of Assessment Instruments


Monthly essays by assessment experts
http://nrc.fye.sc.edu/resources/FYAlist
http://www.sc.edu/fye/resources/assessme
nt/typology.html
Overview of Instruments, Pricing,
Contacts

http://nrc.fye.sc.edu/resources/survey/sear
ch/index.php
Summer Institute on
First-Year Assessment

Intentionally structured to create a
cumulative learning experience for:




Beginners and experts
Institutions of all types
June 25-27, 2006
Asheville, NC
www.sc.edu/fye/events/upcoming.html
References: Attendance
Anderson, C. (2004). Freshman absence-based intervention
at the University of Mississippi. In R. Swing (Ed.), Proving
and improving, Volume II: Tools and techniques for
assessing the first college year (monograph no. 37) (pp.
19 -21). Columbia, SC: University of South Carolina,
National Resource Center for the First-Year Experience
and Students in Transition.
Porter, S. R., & Umbach, P. D. We can’t get there in time:
Assessing the time between classes and classroom
disruptions. Planning in Higher Education, 32(2), 35-40.
References: Retention


Astin, & Oseguera (2005) Degree Attainment Rates at
American Colleges and Universities, revised edition.
Consortium for Student Retention Data Exchange


ACT National Retention Statistics


http://www.act.org/path/policy/reports/retain.html
Noel-Levitz Retention Revenue Calculator


http://www.ou.edu/csrde/index.html
http://www.noellevitz.com/Papers+and+Research/Retention+
Calculator/
National Student Clearinghouse


Ward, J.P. (2003). Using EnrollmentSearch to Track First-Year
Success. In Swing, R.L. (Ed.) Proving and Improving, Volume
II.
www.studentclearinghouse.com
References: Before & After
HERI – UCLA
http://www.gseis.ucla.edu/heri/heri.html
NSSE
http://www.indiana.edu/~nsse/
CSEQ & CSXQ
http://www.indiana.edu/~cseq/
References: Student Voices

Cox & Stirton (2005) Start Simple: The Value
of Simple Assessment Techniques

http://www.sc.edu/fye/resources/assessment/ess
ays/Cox&Stirton-5.18.05.html
Hodges & Yerian (2004) The First-Year
Prompts Project: A Qualitative Research
Study Revisited
 Claffey & Laff (2004) The Promise Audit:
Who’s Promising What to Students?
Both of the above articles are in:
Swing, R. (Ed.) (2004) Proving and
Improving, Volume II: Tools and Techniques
for Assessing the First College Year

References: Student Voices



Harrington, C.E. & Schibik, T.J. (2003)
Reflexive Photography as an
Alternative Method for the Study of
the Freshman Year Experience. NASPA
Journal, 41(1)
University of Southern Indiana
Complete PPT presentation available
at
https://www.usi.edu/depart/instires/reflexphoto.asp
Contact Info
Bradley E. Cox
Randy L. Swing
Coordinator, Research and
Public Information
National Resource Center on
The First-Year Experience &
Students in Transition, USC
Co-Director & Senior Scholar, Policy
Center on the First Year of
College
1728 College Street
Columbia, SC 29208
803-777-6225
[email protected]
400 North Broad Street
Brevard, NC 28712
828. 966.5312
[email protected]