Understanding the
First-Year Experience
Four Ways to Generate DataDriven, Actionable Insights
Steve Wygant, Danny Olsen, Bryce Bunting
Brigham Young University
26th International Conference on the First-Year Experience
Waikoloa, Hawaii, June 2013
Introduction: Brigham Young University
 Private, faith-based institution
in Provo Utah, USA
 30,000+ undergraduates
 Academically competitive:
 Mean HS GPA (2012): 3.81 (high=4.0)
 Mean ACT/SAT (2012): 28/1270
2
Academics
 Strong programs in accounting, business, engineering,
animation, music, etc.
 Ranked #9 in Baccalaureate-origins of U.S. Doctorate
Recipients
 Ranked #68 in US News & World Report 2013 Best
Colleges
3
International focus
 60 languages taught
 75% of students have a second




language
33% enroll in a foreign language
course each semester
7% go on Study Abroad
6% are International Students
Outreach efforts around the
world
4
2012 First Year Class
 2,455 Women
 2,975 Men
 5,430 Total
 55% of
applicants are
admitted
 80% of admitted
enroll
5
Retention
89% retained
(full-time first-time, Fall 2011- Fall 2012)
Graduation
4 year - 31%
6 year - 78%
8 year - 86%
(2004 cohort as of 2012)
6
Athletics
7
A Tale of Two Students
8
Guiding questions
 Why do intelligent, capable students flounder?
 Academic warning/probation
 Fail to achieve goals, live up to potential
 Can we identify students who will flounder?
 Using data obtained before students arrive on campus?
 Using data obtained during the course of the school year?
 How could and should such data be used?
 Individually, and as a campus community, how do
we help students flourish?
 Can data help us identify student motivations, strengths and
challenges in order to connect them with resources they need?
 How would such data be used?
9
As we get going …
 Focus on processes, not specific findings
 Questions you might ask yourselves
 “How would processes like this work with our campus and
our students?”
 “What data do we have that could help us predict student
success?”
 “What other data can we gather that might help us predict
student success?”
 Session as conversation
 Ask questions
 Share your practice
10
Survey instruments used
 Pre-arrival Survey
 High school experiences & practices
 Expectations for college
 Self-descriptions
 End-of-year Survey
 Engagement: work ethic, participation
 Attitudes, practices
 Self-descriptions
 Outcomes, influences
11
Getting and using good data
12
Four ways of looking at data
1. Before-after
•
How do attitudes & behaviors change? Expectations vs. experience?
2. Flourish-flounder
•
How are students who are thriving by the end of the first year
different from those who are not?
3. Over-achieving vs. underachieving
•
How are students who exceed academic expectations different from
those who perform below expectations?
4. Cluster analysis
•
How do sub-groups of students who begin the first year with
different attitudes and motivations differ in behavior and
performance?
13
Before-after analysis
Expected vs. actual hours spent per weekday
(Expected hours from Pre-arrival survey [PA], Actual hours from End-of-year survey [EOY])
PA
EOY
High
Very high
Using internet/phone (not school work)
Extra-curricular activities
Working for pay
Academic work outside of class
Attending classes
Very low
Low
Moderate
14
Before-after analysis
Percent of students who rated themselves as
"Very competent" or "Quite competent"
PA
think about problems in
original ways
EOY Pre
evaluate the credibility of
information sources
EOY Post
explore a variety of ways
to solve a problem
evaluate your own
assumptions, conclusions
and reasoning
15
Flourish-flounder analysis
Flourish - flounder
(End-of-year survey)
Agree (low) - Disagree (high)
(Pre-arrival survey)
Flourishing
Getting
Floundering
strongly somewhat
somewhat
badly
by
… satisfied with my life
… like the kind of person I am
… able to make friends easily
…. know & understand myself
Very low
Low
Moderate
High
Very high
16
Flourish-flounder analysis
Flourishing-floundering
(End of school year)
Agree (low) - Disagree (high)
(on Pre-arrival survey)
Flourishing
Getting
Floundering
strongly somewhat
somewhat
badly
by
BYU was the right choice
I will be able to make friends at BYU
I will do well academically at BYU
Very low
Low
Moderate
High
Very high
17
Flourish-flounder analysis
Flourishing-floundering
(End of school year)
Flourishing
Frequency during first year
strongly
somewhat
Very low
Low
Floundering
Getting
somewhat
badly
by
Adapt study habits & strategies
Persist in doing class work
Procrastinate doing class work
Moderate
High
Very high
18
Underachieving vs overachieving
Comparing students whose first-year GPA was higher
than expected to those whose GPA was lower than
expected. (Expected GPA [EGPA] is based on ACT/SAT and HSGPA.)
 Badly underachieve: GPA 1/2 point or more lower than EGPA
(Example: EGPA=3.75, GPA<=3.25)
 Somewhat underachieve: GPA ¼ to ½ point below EGPA
 Matched: GPA within ¼ grade of EGPA
 Somewhat overachieve: GPA ¼ to ½ point higher than EGPA
 Strongly overachieve: GPA more than ½ point above EGPA
19
Underachieving vs overachieving
Underachieving (GPA < expected)
vs overachieving (GPA > expected)
High school experience
Underachieved
badly
somewhat
Very low
Low
Matched
Overachieved
somewhat
strongly
High
Very high
Flourished (1) or floundered (5)
Hours/week studied last year
Did readings, assignments before class
Academic success: Ability (1) v. effort (5)
Procrastinated doing class work
Moderate
20
Underachieving vs overachieving
Underachieving (GPA < expected)
vs overachieving (GPA > expected)
Hours per typical weekday
in first year
Underachieved
badly
somewhat
Very low
Low
Matched
Overachieved
somewhat
strongly
academic work outside of class
socializing in person
on-line (not school work)/gaming
working for pay
Moderate
High
Very high
21
Underachieving vs overachieving
Underachieving (GPA < expected)
vs overachieving (GPA > expected)
Underachieved
badly
somewhat
Very low
Low
Matched
Overachieved
somewhat
strongly
High
Very high
Complete readings & assignments pre-class
Get feedback before submitting a paper
Work harder than you thought you could
Seek help when you did not understand
Adapt your study habits and strategies
Persist in doing the work required
Procrastinate doing the work needed
Moderate
22
Cluster analysis
 Questions:
 How do sub-groups among first-year students differ in
ability, attitude, motivation?
 How do those differences affect behavior and
performance?
 Identified subgroups using cluster analysis
 Process
 Factor analysis of Pre-arrival survey questions
 Create scales using multiple items from factor analysis
 Use scales to look for clusters
 Art and science
23
Sample items that form scales
High school academic engagement (How frequently did you …)
Adapt your study habits and strategies to help you do better in your classes
Persist in doing the work required to succeed in your classes, even when it was difficult
Seek help when you did not understand important ideas or concepts
Spiritual/social reasons for choosing BYU (How important was … in your decision to choose BYU?)
Getting a college education in a spiritually enriching environment
Opportunity to associate with peers who have values and beliefs similar to yours
Opportunity to develop a greater capacity to do good for others
Social/academic engagement (How important will ... be during your first year at BYU?)
Talking with faculty members outside of class
Participating in study or discussion groups
Discussing ideas from your reading or classes with others outside class
24
Sample scales that distinguish
between clusters
Cluster
AU
Dis
Bal
Very high
Low
Low
High
choosing BYU
Importance of social/
Moderate
Moderate
Low
High
academic engagement in FY
Moderate
Moderate
Low
High
Scale
High school academic
engagement
AF
Spiritual/social reasons for
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
25
Clusters defined
1. Academically focused (AF)
 Very high on academic experience/motivation scale
 Average on other scales
2. Academically undeveloped (AU)
 Low on academic experience/motivation scale
 Average on other scales
3. Disengaged/reticent (Dis)
 Low on every scale
4. Balanced/well adjusted (Bal)
 High on every scale
26
Cluster analysis
Clusters
Importance of college goals
AF
AU
Dis
Bal
Low
Moderate
High
Very high
Socialize
Learn new ideas and perspectives
Enhance your personal spirituality
Grow from being challenged intellectually
Very low
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
27
Cluster analysis
Clusters
First-year academic participation
AF
AU
Dis
Bal
Low
Moderate
High
Very high
Complete readings and assignments
Discuss course-related ideas outside class
Get feedback before submitting work
Work harder than you thought you could
Write multiple drafts of a paper
Very low
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
28
Cluster analysis
Clusters
Weekly hours spent, first year
AF
AU
Dis
Bal
Low
Moderate
High
Very high
Academic work outside of class
Working for pay (on or off campus)
Socializing in person
Using the internet, other than school
Very low
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
29
Cluster analysis
Clusters
Effort and related outcomes
AF
AU
Dis
Bal
High
Very high
Academic effort during the year
Satisfaction with academic effort
Academic success: Ability (1) vs effort (5)
Very low
Low
Moderate
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
30
How do clusters look at the end
of the first year?
Clusters
Goal accomplishment
AF
AU
Dis
Bal
Low
Moderate
High
Very high
Socialize
Learn new ideas and perspectives
Enhance your personal spirituality
Grow from being challenged intellectually
Very low
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
31
How do clusters look at the end
of the first year?
Clusters
Evaluations
AF
AU
Dis
Bal
Low
Moderate
High
Very high
I feel like I belong at BYU
Made the right choice to attend BYU
Flourishing
Would choose BYU again
Very low
AF=Academically focused, AU=Academically undeveloped, Dis=Disengaged, Bal=Balanced
32
Four ways of looking at first-year data
1. Compare student self-descriptions at the
beginning vs end of year
2. Compare students who describe themselves as
flourishing or floundering at the end of the year
3. Compare overachieving vs. underachieving
students
4. Compare pre-arrival clusters of students over the
course of the year
33
To ponder:
Implications for Intentional Action
Academically Academically
Focused
Undeveloped
Disengaged
Balanced
Admissions
Orientation
Seminars
Learning
Communities
Residence Life
Advisement
Curriculum
Assessment
34
Thank you for coming!
Contact:
• [email protected]
• [email protected]
• [email protected]
35
36