PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Today’s Agenda
1.  Introduction
IUPUI and
‘Gateway’ First-Year Initiatives
Promoting First-Year
Student Success
2.  Measuring Progress
Retention and ‘DFW’ Trends
3.  Explaining & Motivating
Students
Using Assessment & Collaboration
To Improve Student Retention
Through Early Identification & Intervention
B104 Introductory Psychology
4.  Collaboration at IUPUI
‘Gateway’ Success Stories
John F Kremer – Psychology
Kate A Thedwall – Gateway
Kenneth EA Wendeln – Business
23rd International Conference on the First-Year Experience
5.  Questions
©2010 KEAW
Forbes:
Best Colleges in the Midwest
23rd International Conference on the First-Year Experience
2
‘Gateway’ First-Year Initiatives
  10 Year Retention Initiative
  55 ‘Gateway’ Gen-Ed Courses
  22,600 Fall Semester Grades
  450 Instructors
  55 Course Coordinators
  Director and Advisory Board
Current Focus:
Early Identification
and Active Intervention
http://www.forbes.com/2009/11/18/best-colleges-midwest-thought-leaders-public-2009_slide_9.html
23rd International Conference on the First-Year Experience
3
23rd International Conference on the First-Year Experience
IUPUI Entering Students
Fall-to-Fall Retention Trends
IUPUI FTFT Students
Retention vs DFW Grades
Fall-to-Fall Retention for
Full-Time Beginners Only
80%
75%
70%
65%
All Entering
DFW Grades for Fall
Gateway Courses
90%
FTFT (F-T Beginners)
0%
FTFT DFW Grades
80%
Transfers
70%
P-T Beginners
60%
50%
15pp Improvement (+27%)
In Fall-to-Fall Retention
50%
45%
40%
F96 F97 F98 F99 F00 F01 F02 F03 F04 F05 F06 F07 F08 F09
Source: KEAW Analysis
of IU Retention Reports 23rd
International Conference on the First-Year Experience
23rd International Conference on the
First-Year Experience
5
-10%
-20%
60%
55%
4
-30%
FTFT Retention
-40%
40%
-50%
30%
-60%
20%
-70%
10%
-80%
0%
Cohort F96 F97 F98 F99 F00 F01 F02 F03 F04 F05 F06 F07 F08 F09
Source: KEAW Analysis
of IUPUI DFW Reports 23rd
International Conference on the First-Year Experience
-90%
6
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 1
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
IUPUI All Entering Students
Retention vs DFW Grades
Gateway Courses
DFW Trends - All Students
DFW Grades for Fall
Gateway Courses
Fall-to-Fall Retention for
All Students Entering in Fall
0%
-5%
90%
All Gateway DFW Grades
80%
-15%
Fall 07
60%
-35%
Fall 06
-45%
40%
30%
-55%
-65%
20%
-75%
10%
-85%
0%
-95%
F96 F97 F98 F99 F00 F01 F02 F03 F04 F05 F06 F07 F08 F09
Source: KEAW Analysis
of IUPUI DFW Reports 23rd
International Conference on the First-Year Experience
Gateway Courses
DFW Trend by Department
# Grades
Fall 02-09
#22637
29.2%
#22376
31.5%
#23041
32.1%
Fall 05
#23345
31.5%
Fall 04
#24124
31.1%
Fall 03
#24091
29.7%
Fall 02
#24089
5%
Source: KEAW Analysis
of IUPUI DFW Reports 23rd
10%
50
45
40
35
30
25
20
15
10
5
0
COMM #1503
09
08
PHIL #620
07
06
05
04
HIST #1649
03
02
ENG #2552
Source: KEAW Analysis 23rd
of IUPUI DFW Reports
15%
20%
25%
30%
35%
International Conference on the First-Year Experience
8
Gateway Courses
Fall 2009 DFW by Student Type
MATH #4370
LEARN COMM
#2378
Ranked by
Average
ANTH #567
DFW%
Fall
Fall
Fall
Fall
Fall
Fall
Fall
Fall
27.2%
0%
7
Total # of Grades
#22099
Fall 08
-25%
Retention
All Entering
Students
23.6%
Fall 09
70%
50%
All Gateway DFW%
Semester
MATH #4112
PSY #1831
45
40
35
30
25
20
15
10
5
0
ANTH #504
LEARN COMM
#2544
CHEM #1246
BIOL #1077
PHIL #509
Σ Gateway #23225
BIOL #1097
# Grades
Fall 09 by Dept
Σ Gateway# 22099
PSY #1803
SOC #778
CHEM #1456
DFW% by Type
Fall 09 All
BUS #1869
COMM #1444
SOC #881
ENG #2400
BUS #1625
International Conference on the First-Year Experience
9
Psychology B104
DFW Trends
Source: KEAW Analysis
of IUPUI DFW Reports 23rd
HIST #1461
Fall 09 FTFT
Fall 09 Other
International Conference on the First-Year Experience
10
Can We ….. These Students?
…..Describe
Semester
Fall
Fall
Fall
Fall
Fall
Fall
Fall
Fall
2002 2003 2004 2005 2006 2007 2008 2009
…..Explain
# Grades
1516 1692 1559 1463 1365 1222 1377 1434
…..Predict
DFW %
26.8
31.6
35.9
32.7
45.9
34.3
32.0
22.9
…..Change
Example:
B104: Introductory Psychology
23rd International Conference on the First-Year Experience
23rd International Conference on the
First-Year Experience
11
23rd International Conference on the First-Year Experience
12
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 2
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Possible Reasons
for DFW Grades
What Group has the Lowest
Percentage of DFW Students?
  Prior to IUPUI
o 
High School GPA and SAT/ACT
  Personal Preparation, Attitudes and Perceptions
o 
o 
Test Preparation Skills
Perseverance in Studying
o 
Interest in Doing Homework
o 
Expect Success in Study Efforts
Number of Hours This Semester
Homework Completion
  Current Non-Academic Behavior
o 
Hours Working
o 
Life Stress
13
What Group has the Lowest
Percentage of DFW Students?
Attend
Class
Did
HWK
Pass
Tests
No
No
No
Yes
No
No
Yes
Yes
No
Yes
Yes
Yes
Nonattenders
Noncompliers
Low
Performers
Drop Outs
10%
Did
HWK
Pass
Tests
?
No
No
No
?
Yes
No
No
?
Yes
Yes
No
?
Yes
Yes
Yes
Drop Outs
23rd International Conference on the First-Year Experience
% of
DFWs
Attend
Class
Nonattenders
Noncompliers
Low
Performers
  Current Academic Behavior
o 
o 
% of
DFWs
23rd International Conference on the First-Year Experience
23rd International Conference on the First-Year Experience
Explaining Dropouts (10%)
  Are Dropouts related to HS GPA
or SAT/ACT?
15
Explaining Dropouts (10%)
23rd International Conference on the First-Year Experience
  Person A
  Working 40 hrs/wk
  No quantitative data explained these
students. Qualitative interviews
suggested the following
  Another Course: 2 hrs/day
  Husband sent to jail
  Can’t afford internet connection
  Severe Stress
  Nothing related to the course
  Working 40 hrs/wk
  Person B
  Taking 6-15 hours
  Working 42 hrs/wk
  Plus: One course takes 2-3 hrs/day; &/or
  Brother committed suicide
  Plus: Major life event
  Sleeps 10-12 hrs/day: Migraine pain
  Nothing related to the course
  Cannot Predict Dropouts
23rd International Conference on the
First-Year Experience
16
Examples Dropouts (10%)
  NOT related to HS GPA or SAT/ACT
23rd International Conference on the First-Year Experience
14
17
23rd International Conference on the First-Year Experience
18
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 3
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Support for Stressed Students
What Group has the Highest
Percentage of DFW Students?
Create Flexible Course Structure
C: Tests available 10 hrs/day, 7 days/wk during 3week period
C: Count highest of 3 test scores
C: If miss a class, may substitute any of 30 other
sections: JagTag
Actively Reach Out
C: Course Design
M: Course Mentoring
U: University Support
M: Contact & Understand
U: Advising?
% of
DFWs
Attend
Class
Did
HWK
Pass
Tests
?
No
No
No
?
Yes
No
No
?
Yes
Yes
No
10%
Yes
Yes
Yes
Nonattenders
Noncompliers
Low
Performers
Drop Outs
& Policies
23rd International Conference on the First-Year Experience
19
Describing Non-compliers
% of
DFWs
Nonattenders
Noncompliers
Low
Performers
40%
Drop Outs
10%
Did
HWK
Pass
Tests
No
No
No
Yes
No
No
Yes
Yes
No
Yes
Yes
Yes
  Are non-compliers related to HS
GPA OR SAT/ACT?
21
Explaining Non-compliers (40%)
  Motivational factors explained
ONLY 21% of homework completion
22
Explaining Non-compliers (40%)
Homework in High School
o  HS = 4 hrs/wk for ALL courses
  Hours working
  Perseverance in Studying
o 
Two separate surveys of 400 students each
o 
Survey of 15 high school, social studies teachers
o  B104 & all SOS courses = 4 hrs/wk for each
  Perception of Quality of Homework
o  15 hour load = 20 hrs/wk
  Implies many factors differing
remarkably across students
23rd International Conference on the
First-Year Experience
23rd International Conference on the First-Year Experience
  Key Factor:
  NOT related to HS GPA OR SAT/ACT
23rd International Conference on the First-Year Experience
20
Explaining Non-compliers (40%)
Attend
Class
23rd International Conference on the First-Year Experience
23rd International Conference on the First-Year Experience
o  College requires approximately 5X the amount
of studying outside of class than high school
23
23rd International Conference on the First-Year Experience
24
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 4
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Predicting Non-compliers (40%)
  Homework predicts
test scores
(r=.4-.5)
  Homework in the
1st week of the
semester predicted
total homework
completed for the
semester(r2 = .62)
23rd International Conference on the First-Year Experience
Drop Outs
Attend
Class
Did
HWK
Pass
Tests
25%
No
No
No
40%
Yes
No
No
?
Yes
Yes
No
10%
Yes
Yes
Yes
Pass
Tests
?
No
No
No
40%
Yes
No
No
?
Yes
Yes
No
Drop Outs
10%
Yes
Yes
Yes
23rd International Conference on the First-Year Experience
27
26
23rd International Conference on the First-Year Experience
28
10 Minute Student Survey
Beginning of the semester asked all
students about completing the course
  NOT related to HS GPA or SAT/ACT
  No quantitative data predicted these
students.
  100% of students on first day of class
stated they were definitely or very likely
to complete the course
  Unable to interview even with extreme
effort
  Yet, 25% do not complete the course
  Students are Unavailable and do not
attend for Unknownreasons
  Why?
  A few reasons?
WHO should be responsible for
contacting chronic non-attenders?
23rd International Conference on the
First-Year Experience
Did
HWK
  Are non-attenders related to HS
GPA or SAT/ACT?
Explaining Non-attenders (25%)
23rd International Conference on the First-Year Experience
Attend
Class
Explaining Non-attenders (25%)
% of
DFWs
23rd International Conference on the First-Year Experience
% of
DFWs
Nonattenders
Noncompliers
Low
Performers
25
What Percentageof DFWs
Were Non-attenders?
Nonattenders
Noncompliers
Low
Performers
What Percentageof DFWs
Were Non-attenders?
  Many different reasons?
29
23rd International Conference on the First-Year Experience
30
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 5
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Understanding Non-attenders
to Motivate Them
Follow-up Student Survey
Why would you NOT successfully
complete the course?
Need Early & Active Interventions
C: Motivate on 1st day of class
  80 students who did not take all exams
M: Contact & Understand
  51 different reasons (listed are >5%)
C: Course Design
M: Course Mentoring
U: University Support
& Policies
  Personal reasons (job – 8%)
U: Threat of Administrative Withdrawal
  Test Taking (nervous, poor test taker – 15%)
U: Revise Early Warning System
  Study behaviors (didn’t do HWK or didn’t prepare
well/didn’t know material – 16%)
U: Who (advisors?) follows-up on early
  Motivation (lazy, skip class – 16%)
23rd International Conference on the First-Year Experience
31
Predicting the Incompleters
23rd International Conference on the First-Year Experience
Non-attenders and Non-compliers
  Students who do not complete the class can be
identified in first 3-4 weeks of the semester by:
When can we predict the students
who will not complete their work?
  Attendance &
  HWK completion (r2 = .42)
  6 weeks
  Students who do not complete the class can be
identified in first week of the semester by:
  4 weeks
  Attendance &
  2 weeks
  HWK completion (r2 = .26)
  1 week
Attendance & homework completion
QUICKLY predict the Incompleters
33
Predicting the Incompleters
23rd International Conference on the First-Year Experience
  Students who do not complete the class can be
identified in first 5 days of the semester by:
  attendance &
Create Relevance in Course Content
C: Class pts HWK & timely finish
C: Interesting HWK with validity
  Homework completion
  54% successfully completed 2 points of homework
C: Application to students’ lives
C: Course Design
M: Course Mentoring
U: University Support
& Policies
  13% successfully completed 1 point of homework
Early and Multiple Interventions
  7% completed 1 HWK but below criteria
M: Interview, identify obstacles, problem solve, set
goals, follow-up
  26% did not complete any homework
M: Emphasize weeks 1-4
Attendance & homework completion
QUICKLY predict the Incompleters
23rd International Conference on the
First-Year Experience
34
Developing Motivation
Non-attenders and Non-compliers
23rd International Conference on the First-Year Experience
32
Predicting the Incompleters
Non-attenders and Non-compliers
23rd International Conference on the First-Year Experience
warning
U: Change Ws to WX (not official grade)?
U: Administrative Withdrawal tied to homework
completion, response to an early warning system?
35
23rd International Conference on the First-Year Experience
36
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 6
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Results for Incompleters I
Non-attenders and Non-compliers
Did these changes in course
structure, assistance, & policy
increase homework completion?
Some notable exceptions:
  Homework completion
dependent on faculty who are
teaching the course
  Moderate changes in homework
  1st change recommended homework = 3
  Actual homework completion = 2.74
  2 faculty had the lowest 6
sections with a 2.2 homework
rate
  3 faculty had 9 of the highest 11
sections with a 3.1 homework
rate
  Excluding online courses
23rd International Conference on the First-Year Experience
Results for Incompleters II
Non-attenders and Non-compliers
37
Results for Incompleters III
Non-attenders and Non-compliers
23rd International Conference on the First-Year Experience
Results for Incompleters IV
Non-attenders and Non-compliers
Some notable exceptions:
Some notable exceptions:
  Homework completion
dependent on when class was
offered
Athlete section (1 of 28)
  At the top in attendance
  2nd highest in homework completion
  Sections outside of 7:30 – 4:30
M-R had 6 of 9 lowest sections
  The highest course completion rate
  2 online (1.03)
  2nd highest ABC (pass) rate
  4 on-campus (2.25)
23rd International Conference on the First-Year Experience
  2nd lowest test scores
39
  Athletic Advisors
40
  University/State/
National Policy
  Immediate feedback
to advisors
  Talk to athletes
  Consequence
  Course Structure
  Instructor
  Student Assistants
  Study hall
  Talk to Coaches
  Advisors
  Coaches – If consequences
  Former students
23rd International Conference on the
First-Year Experience
23rd International Conference on the First-Year Experience
Multiple Factors for Success
Why Athletes Better??
23rd International Conference on the First-Year Experience
38
Many are
Required
&
Differ by
Student
  Support Group
  Former Students
41
23rd International Conference on the First-Year Experience
42
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 7
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
What Percentageof DFWs
Were the Low Performers?
Nonattenders
Noncompliers
Low
Performers
Drop Outs
Describing All DFW Students
% of
DFWs
Attend
Class
Did
HWK
Pass
Tests
25%
No
No
No
40%
Yes
No
No
?
Yes
Yes
No
10%
Yes
Yes
Yes
23rd International Conference on the First-Year Experience
Nonattenders
Noncompliers
Low
Performers
Drop Outs
43
Explaining Low Performers (25%)
% of
DFWs
Attend
Class
Did
HWK
Pass
Tests
25%
No
No
No
40%
Yes
No
No
25%
Yes
Yes
No
10%
Yes
Yes
Yes
23rd International Conference on the First-Year Experience
Explaining Low Performers (25%)
SAT Predicts Total Points?
(r =.20)
#32
  Are low performers related to HS
GPA or SAT/ACT?
44
HS GPA Predicts Total Points?
(r =.26)
#35
r= .35
r= .36
r=-.16
r=-.10
#30
#23
SAT & HS GPA predict the High Performers,
But NOT the Low Performers & Incompleters
23rd International Conference on the First-Year Experience
45
Explaining Low Performers (25%)
  Study Skill factors (primarily)
determine 58% of test performance
o  Test Preparation Skills = Best Predictor
23rd International Conference on the First-Year Experience
10 Minute Student Survey
How would you rate your test
preparation skills?
o  Homework Points
  28% said poor (20%), very poor
(6%), or terrible (2%)
o  Hours Working
  An additional 24% said fair
o  Expectancy for Study Success
46
  HS GPA or SAT/ACT = 32%
Cannot change HS GPA or SAT/ACT,
But CAN change test prep skills
23rd International Conference on the First-Year Experience
23rd International Conference on the
First-Year Experience
47
23rd International Conference on the First-Year Experience
48
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 8
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
10 Minute Student Survey
10 Minute Student Survey
How will you prepare for your first
test?
How will you prepare for your first
test?
  Read the book TWICE
  Read the book TWICE ~ 20%
  LESS than this
  LESS than this ~40%
  MORE than this
  MORE than this ~40%
23rd International Conference on the First-Year Experience
49
Developing Study Skills
Pre-College GPA and SAT/ACT is
limited in predicting B104 success:
C: Develop study aids
Flashcards, memory outlines, practice tests,
Q/A text boxes
C: Drop a test
C: Immediate Feedback
50
Putting It All Together – I
Low-risk skill-building opportunities
C: Multiple test opportunities
23rd International Conference on the First-Year Experience
C: Course Design
M: Course Mentoring
U: University Support
& Policies
Provide a skill-building discipline
  Predicted only 1 of the 4 groups of
students: Low Performers on Tests
  Had little relationship to the DFW
grades of 75% of the students in
the course
  SAT and HS GPA have low r with
total course points
M: Match study method with student interest, set
goals, follow-up
U: Nothing?
23rd International Conference on the First-Year Experience
51
Putting It All Together – II
Students Come to College with:
  Very low failure rates in HS
  High Expectations for:
Students Pass even though:
  Completing the course (100%)
  Course grades (95% As or Bs)
  Low levels of HWK activity
  Low Expectations for:
  Low levels of HWK completion
  Completing homework
  Low test preparation skills
  Required test preparation skills
Thus, LOW ACADEMIC SKILLS
23rd International Conference on the
First-Year Experience
52
Putting It All Together – III
Culture to Schools: Pass Them
23rd International Conference on the First-Year Experience
23rd International Conference on the First-Year Experience
53
23rd International Conference on the First-Year Experience
54
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 9
PROMOTING FIRST-YEAR STUDENT SUCCESS
Using Assessment & Collaboration To Improve Student Retention
Through Early Identification & Intervention
Changing
Marginally Motivated Students
Putting It All Together – IV
Depends on
Type of Student
And many students have
LOW PERSEVERANCE when
facing academic obstacles
  35% said facing these academic
obstacles would get to them and
probably prevent them from
doing their best
  They think they…
  Course is too difficult for them
Low Performers –
semester
 
Non-compliers –
more difficult
 
Non-attenders –
many semesters
 
55
4 courses, 2 years
C:
Class, Course
&Curriculum
M: Mentoring &
Support Programs
U:
University
Administrative
Policies
  Must use all Areas
  Areas must be
coordinated
  Some areas more
important for some
problems
23rd International Conference on the First-Year Experience
+
A Silver Bullet
Is NOT a Model for Success
56
Additive Model
Can Lead to Improvement
✗
In the Silver Bullet model,
you search to find the single straight
forward solution that will have a
dramatic impact on student learning.
23rd International Conference on the First-Year Experience
Dropouts - easy
 
  Most have chronic
habits that require
many semesters of
integrated work
  Can’t do it
23rd International Conference on the First-Year Experience
 
Levels of
Intervention
Course Design

Faculty
"
Mentoring
"
Advising
"
Coaching

Administrative
Policy
For the Additive model,
each improvement increases the
amount of student learning.
57
Multiplicative Model
Recognizes ALL Key Factors
23rd International Conference on the First-Year Experience
58
Improving Retention from the
‘Inside Out’
IUPUI
Meta-Course
Mentoring/Support
Courses &
& Policies
Curriculum
My
Your
Class
Class
In the Necessary but not Sufficient
or Multiplicative model,
improvement in student learning occurs only when
ALL of the necessary steps are present and active.
23rd International Conference on the First-Year Experience
23rd International Conference on the
First-Year Experience
59
http://gateway.uc.iupui.edu
[email protected]
[email protected]
[email protected]
23rd International Conference on the First-Year Experience
60
J Kremer, K Thedwall & K Wendeln
June 2010 © KEAW Page # 10