Developing, Implementing, and Assessing an Early Alert System
Annual Conference on the First Year Experience (2010)– Session CI 163
Dale R. Tampke, Ph.D.
Dean, Undergraduate Studies
University of North Texas
940.565.4321
[email protected]
1
The Main Points
Our Context ‐ UNT
Early Alert as a Concept
Project Scope (the tech‐y part)
Building Advocacy
How EARS works
– End‐user
– Responder
• Data from AY 2008‐2009 (and what we’ve learned so far)
• Enhancement plans
• Questions
•
•
•
•
•
2
The University of North Texas
• Located in Denton, TX
• Enrollment – 36,000
• High Research university offering 99 bachelor’s, 104 master
master’ss and 49 PhD and 49 PhD
programs
• Faculty – 1007 full‐time, 430 part‐time
• FTIC retention – 75%
3
3
1
Early Alert as a Concept
• Grounded in literature on undergraduate retention
– Student behavior can predict attrition
– Early intervention can change outcomes
Early intervention can change outcomes
• First efforts were course‐centered
– Poor performance
– Excessive absences
(Think “mid‐term” grades)
4
More on the Early Alert Concept
• Expansion to campus‐wide availability
– Include psycho‐social concerns
– Web front end
– E‐mail back end
E mail back end
– Authentication varies
– Integration varies
• A common issue: How many faculty use the system?
5
Our Idea…
•
•
•
•
•
•
•
Integrate with student system
Start with a focus on faculty (make it easy for them)
Designate a central receiver of the data
Expand beyond Expand beyond “academic”
academic issues
issues
Have a ready referral
Begin a personal, caring conversation
Increase student success and persistence
6
2
Building the System
• Include stakeholders
– Students (8 from office staffs)
– Faculty (12 from Arts and Sciences)
– Academic Advisors (10 from all colleges)
– Student Services (15 areas)
• Get their feedback at the conceptual stage
• Adopt a good idea
• Faculty test group
7
The Set Up
• PeopleSoft Campus Solutions 9.0
• Early Alert Referral System (EARS) 1.0 (available from audit roll and class roll)
• Instructors of record receive an e‐mail reminding them of EARS
• Accessed through the faculty portal (The “Faculty Center”)
• Nightly report delivered to Academic Readiness
• Follow‐up within one day of receiving
• Points of contact in each college
8
Accessing Early Alert – From the Faculty Center
9
3
To the class roster …
10
From the Class Roster…
Faculty click the icon
11
To the Early Alert Form
Data pre-populated
Drop down boxes
12
12
4
Reasons for Referral
(what’s on the drop down menu)
• Poor class attendance
• Poor performance on quizzes/exams
• Poor performance on writing assignments
• Does not participate in class
l
• Difficulty completing assignments
• Difficulty with reading
• Difficulty with math
• Sudden decline in academic performance
• Concerns about their major
•
•
•
•
•
•
•
•
•
•
College adjustment issues
Financial problems
Physical health concerns
Mental health concerns
Alcohol or substance use concerns
Roommate difficulty
Disruptive behavior
Absent from work
Student needs veterans assistance
Other concerns (text box)
13
Other features
• Relationship to student
–
–
–
–
–
–
–
Professor, instructor
Teaching assistant, teaching fellow
Academic Advisor
Mentor
Department administrator
Campus Employer
Club, organization advisor
• “I have had a conversation with the student”
• Send a copy of the referral to the student (via e‐mail)
14
What Happens Next…
• Review report every morning
– E‐mail prompt
– Excel file
– Send an e‐mail acknowledgement to faculty member (manual)
• Includes following information
–
–
–
–
–
Demographics
Student ID
Faculty member’s name
Course
Reason(s) for referral
15
5
Follow‐up
• First responders – Routine referrals
– Residence hall staff
– Course Interventionists
• More serious issues
– Academic Readiness Advisors
– Academic Advisors
– CARE team
– Counseling, Health Center
16
Follow‐up (more)
•
•
•
•
•
•
•
E‐mail, call, text, visit
Caring conversation (no scolding)
Emphasize mattering
Resources
Self‐efficacy
Focus on academic success
Follow‐up2 (we need to get better at this)
17
YEAR 1 DATA
18
6
Alert Frequency by Week in Semester
70
60
50
40
30
20
10
0
1
2
3
4
5
6
7
Fall (n=255)
8
9
10
11
12
13
14
15
Spring (n=280)
19
Demographics
Characteristic
Fall (n=255)
Spring (n=280)
Campus Women
Men
49.0
51.0
50.5
49.5
56.6
43.4
White
African‐American
Hispanic/Latino
Asian/Pacific Islander
Asian/Pacific Islander
Native American
Non‐resident
Other
63.9
17.6
11.8
3.9
39
0.4
1.6
0.8
63.9
18.8
10.1
6.5
65
0.7
2.9
0.7
64.9
12.6
11.2
4.8
48
0.8
4.5
1.2
First Time in College
Continuing Undergraduate
Graduate
10.6
83.4
6.1
14.8
79.1
6.0
10.9
68.9
20.2
20
Reasons for Referral
Fall (%)
Spring (%)
Poor class attendance
First Reason Cited
56.5
65.7
Poor performance
29.1
17.9
Other reason
10.2
13.9
Does not participate in class
1.6
1.4
Mental/physical
M
l/ h i l health h lh
concerns
1.4
14
0.7
07
Concerns about major
0.8
‐‐
‐‐
0.4
0.4
‐‐
Disruptive behavior
Alcohol/Substance use concerns
21
7
Who is Referring?
Relationship to student
Fall (n=255)
Spring (n=280)
Professor or instructor
86.3 %
91.4 %
Teaching assistant
T hi
it t
12.2 %
12 2 %
82%
8.2 %
Academic advisor
1.6 %
0.4 %
22
Outcomes
• Literature suggests early intervention impacts:
– Student success
– Student persistence/progression
• GPA (semester and cumulative)
GPA (
t
d
l ti )
• Re‐enrollment
• Use a within‐group comparison
23
Preliminary Findings
• Fall
o FGPA – 1.39
o CGPA – 1.94
o Persistence – 70%
• Spring
o SGPA – 1.46
o CGPA – 2.10
o Persistence – 64%
24
8
Preliminary Findings (course grades)
Fall
•
•
•
•
•
•
•
•
Spring
• A’s – 4.4%
• B’s – 12.9%
• C’s – 12.8%
• D’s – 9.7%
• F’s – 38.9%
• I’s – 0.9%
• Drops – 17.7%
• Other ‐ 2.7%
A’s – 3.4%
B’s – 5.9%
C’s – 11.9%
D’s – 12.3%
F’s – 43.0%
I’s – 1.3%
Drops – 21.7%
Other – 0.0%
25
Contact Types (frequencies)
• Fall (n=255)
– Faculty – 13.3%
– Staff – 31.4%
– EE‐mail
mail only only – 65.9%
• Spring (n=280)
– Faculty – 12.1%
– Staff – 62.1%
– E‐mail only – 31.8%
(the percentages exceed 100 because of duplicated contacts)
26
Outcomes by Contact Type (all)
Term GPA
Persistence
(% re‐enrolling)
Fall
Faculty (n=25) 2.15
83.3
Staff (n=80) 1.64
74.7
E‐mail
E mail only (n=168)
onl (n 168) 1.26
1 26
67.9
67 9
Spring
Faculty (n=34) 1.67
73.5
Staff (n=174) 1.48
66.7
E‐mail only (n=89) 1.39
58.4
27
9
What we have learned
1. Including faculty in the design was critical
2. Linking to class roll, self‐populating made it easier for faculty to use
3. Faculty generally focus on course‐related issues
4. Personal faculty contact is the most effective follow up
5. E‐mail contact by itself is not effective
6. Some positive effect on success and persistence based on type of contact
7. Timing of alert has no apparent effect on success or persistence
8. Tracking confirmed contacts needs improvement
9. EARS is not a “large class” solution
28
Plans for EARS 2.0 Available campus wide, secure portal
E‐mail confirmation for referrals
Group alert capability
Include SRI scores
Data warehousing
Notification for multiple alerts for the same student
• Use for mid‐term reporting (e.g athletes, other scholarship students)
•
•
•
•
•
•
29
References
•
Bowen, E., Price, T., Lloyd, S., & Thomas, S. (2005). Improving the quantity and quality of attendance data to enhance student retention. Journal of Further and Higher Education, Vol. 29 (4), 375‐385.
•
Eimers, M. (2000). Assessing the impact of the early alert program. AIR 2000 Annual Forum Paper. (ERIC Document Reproduction Service No. ED446511) Retrieved February 28, 2009, from ERIC database.
•
Fischman, J. (2007, October 29). Purdue uses data to identify and help struggling students. Chronicle of Higher Education Online Retrieved May 15 2009 from
Chronicle of Higher Education Online, Retrieved May 15, 2009 from http://chronicle.com/daily/2007/10/530n.htm. •
Geltner, P., & Santa Monica Coll., CA. (2001). The characteristics of early alert students, Fall 2000. (ERIC Document Reproduction Service No. ED463013) Retrieved February 28, 2009, from ERIC database.
•
Hudson, W. (2006). Can an early alert excessive absenteeism warning system be Effective in retaining freshman students? Journal of College Student Retention, Vol. 7(3‐4), 217‐
226. 30
10
More references
•
Kelly, J. & Anandam, K. (1979). Computer enhanced academic alert and advisement system. (ERIC Document Reproduction Service No. ED216722) Retrieved February 23, 2009, from ERIC database.
•
Richie, S. & Hargrove, D. (2005). An analysis of the effectiveness of telephone intervention in reducing absences and improving grades of college freshmen. Journal of College Student Retention, Vol. 6(4), 395‐412. •
The Hanover Research Council. (November 2007). Universal tracking and early warning The Hanover Research Council (November 2007) Universal tracking and early warning
systems for student retention. Washington, DC: Author.
•
The Hanover Research Council. (May 2008). Intrusive advising and large class intervention strategies: A review of practices. Washington, DC: Author.
•
Wasley, P. (2007, February 9). A secret support network. Chronicle of Higher Education, 53(23), A27. 31
QUESTIONS?
32
11