Early
Grades:
EarlyWarning
Warning Grades:
Heeding
Warning
Heeding the
the Warning
William Polley, Ph.D.
Angela Lynn, Ph.D.
Fall 2012 IACRAO Conference
Session 25.25
Institutional Profile
• Public, Four-Year University in West Central IL
– Residential campus in Macomb, IL
– Non-residential campus in Quad Cities/Moline, IL
• Founded as a Teacher’s College in 1899
• Offer 66 baccalaureate degrees, 36 master’s
degrees and 1 doctoral degree
• Total enrollment Fall 2012: 12,205 students
(Undergraduate: 10,263, Graduate: 1,942)
Early Warning Background
• Originated in Fall 1982
• Initial target population
– Freshmen, sophomores, transfers, academic probation
– Notices to students earning less than C grades
• Current population
– Undergraduates in regularly scheduled on-campus courses
• Continued purpose
– Timely communication
• Improve student performance
• Increase retention
Purpose of Study
• Ad hoc committee of Faculty Senate established
• Membership
– Voting: 6 faculty & 2 students
– Ex-officio: 1 academic advisor & registrar
• Charge of committee
–
–
–
–
Identify ways to improve faculty participation
Determine best timing of early warning grades
Consider conversion to midterm grades
Address any other appropriate issues
Plan for the Study
• Consider timing issue
• Review early warning and grade data from Fall
2011 and retention from fall to spring
• Survey faculty and students for attitude
toward early warning system
• Consider midterm grades in light of grade data
and surveys
• Any obvious ways to bring system up to date?
Timing of Early Warning Grades
• Easiest issue to address
• Institutional processing vs. individual notices
– Early enough to affect change
– Late enough to have graded work
• Early warning notices primarily dictated by last
day to withdraw
– Online screen open to faculty in 6th week of semester
• Two weeks before notices sent to students
– Notices sent to students at start of 8th week of
semester
• Two weeks before withdrawal date
Early Warning System
• Specific grades are reported only for C- and
below.
• C or better is reported as “*”.
• If the instructor does not submit, it is reported as
“N/A”.
• Student and adviser receive e-mail alert if any Cor below.
• No e-mail alert if all “*” or “N/A”.
• Student can see their report on STARS even if all
are “*” or “N/A”.
Early Warning and Retention
• The wrong question to ask:
– What is the retention rate among students
receiving an early warning (C- or below)?
– Too easy. You already know the answer.
Early Warning and Retention
• Better questions More informative answers
• What is the effect of receiving an early
warning report by e-mail?
– The “Hidden N/A” problem
• Controlling for academic factors (GPA,
whether regularly admitted, whether they
maintained a C or better in any class), does a
“hidden N/A” matter?
Early Warning and Retention
• A “hidden N/A” does have a statistically
significant impact on fall to spring retention.
• Our statistical model (probit regression)
predicts:
– Students with a 2.0 GPA and who had one “C or
better” (“*”) on their report but had a “hidden
N/A” had a fall to spring retention rate of 83%
compared to 87% for students with no “hidden
N/As.”
Early Warning and Retention
• Statistically significant, but not extremely large
– About 19 students for a school our size
• Much larger effect from one of our control
variables
– Did they maintain at least one C or better?
Early Warning and Retention
• Students who maintain at least one C or
better had a fall to spring retention rate of
90%.
• Students who did not maintain a C from
midterm to final had fall to spring retention
rates much lower—around 56%.
• These are overall percentages, independent of
GPA (which is the most statistically significant
predictor of retention included in our model).
Faculty and Student Attitudes
• Surveyed students and faculty
• Focused on faculty who taught freshmen
– About 70% of responding faculty (N=190)
• Most questions on a 5 point Likert scale from
1=Strongly Disagree to 5=Strongly Agree
Faculty Attitudes
• Among faculty teaching freshmen…
– 43% can determine midterm grades with a high
degree of precision.
– 44% can distinguish students in serious difficulty,
but not specific letter grades at midterm.
• Full professors and faculty not teaching
freshmen were more likely be in the 2nd group
(can’t determine specific letter grade).
Faculty Attitudes
• Question A: The nature of my course is such that midterm grades or
early warning grades are unnecessary in my opinion.
• About 19% of faculty respondents either agree or strongly agree.
Faculty Attitudes
• Question B: Midterm grades or early warning grades are a good
idea in general, but my students don’t need them because they
should know where they stand.
• About 39% of faculty respondents either agree or strongly agree.
Faculty Attitudes
• Question C: I am aware of students of mine who have dramatically
improved their performance after receiving a low early warning
grade.
• About 41% of faculty respondents either agree or strongly agree.
Faculty Attitudes
• Question D: Informing students of their performance throughout
the semester, including giving early warning grades, is something
that I consider to be a priority.
• About 66% of faculty respondents either agree or strongly agree.
Faculty Attitudes
• Question E: If no students are earning less than a C at the time
when early warning grades are due, then I am less likely to log into
the system and submit the early warning grade roster.
• About 26% of faculty respondents either agree or strongly agree.
Faculty Attitudes
• Question F: The culture of my department/college
encourages the submission of early warning grades.
• About 59% of faculty respondents either agree or
strongly agree.
Faculty Attitudes: Major Findings
• 63% (12 of 19) of faculty who did NOT submit
early warning grades responded with
agreement that they are unnecessary
(question A).
• 50% (9 of 18) of faculty who did NOT submit
early warning grades responded with
agreement that midterm grades are a good
idea in general but that their students should
know where they stand (question B).
Faculty Attitudes: Major Findings
• All 11 of faculty who strongly agree with the
statement that they are aware of students
who have improved as a result of early
warning grades did submit early warning
grades.
• All 41 of faculty who strongly agree with the
statement that communication is a priority
submitted early warning grades. (94%
combined agree and strongly agree).
Faculty Attitudes: Major Findings
• Only 54% (7 of 13) of those who strongly
disagree that communication is a priority
submitted early warning grades.
• Submission rates decreased monotonically
with increasing agreement with question E.
Student Attitudes
• The following slides report the survey results
for freshmen only.
– Retention focus
• Number of responses from students by class
were 79, 81, 138, and 148 (Fr., So., Jr., Sr.)
• Upperclassmen responses more reflective of
their experience
• Summary of differences between freshmen
and upperclassmen at the end of this section
Student Attitudes
• Question A: Receiving early warning grades is
important to me.
Student Attitudes
• Question B: If I received an early warning
grade of C- or below, I would contact the
instructor of that course as soon as possible.
Student Attitudes
• Question C: I am frequently surprised by my
early warning grades.
Student Attitudes
• Question D: If I do not receive an early warning
grade in a course, I have other ways of
determining my grade (e.g. Western Online).
Student Attitudes
• Question E: I would like to receive a specific
letter grade in each course at midterm.
Student Attitudes
• Question F: If I received an early warning
grade of C- or below, I would be likely to drop
the course.
Student Attitudes
• Question G: If I received an early warning grade
of C- or below, I would consider dropping the
course, but am more likely to remain in the
course.
Student Attitudes
• Question H: If I received an early warning grade
of C- or below, I would likely be contacted by my
academic adviser or other University staff
member regarding my grades.
Student Attitudes
• Question I: If I received an early warning grade of C- or
below, I would seek additional help from tutoring labs,
the Office of Academic Services, my academic adviser,
and other resources as necessary.
Student Attitudes
• Question J: If I received an early warning
grade of F, I would be likely to drop the course.
Student Attitudes: Major Findings
• A. Freshmen are more likely to say that
“Receiving early warning grades is important
to me.”
• E. Freshmen are more likely to say, “I would
like to receive a specific letter grade in each
course at midterm.”
Student Attitudes: Major Findings
• H. Freshmen are more likely to say, “If I
received an early warning grade of C- or
below, I would likely be contacted by my
academic adviser or other University staff
member regarding my grades.”
Student Attitudes: Major Findings
• I. Freshmen are more likely to say, “If I
received an early warning grade of C- or
below, I would seek additional help from
tutoring labs, the Office of Academic Services,
my academic adviser, and other resources as
necessary.”
• J. Freshmen are less likely to say, “If I
received an early warning grade of F, I would
be likely to drop the course.”
The Secret of Our Success: Participation
• First, do no harm!
• Voluntary system
• Participation by faculty teaching freshmen is
reasonably good
• Lower participation in classes where early
warnings are probably not critically needed
(physical education activities, music
ensembles, and the like)
Midterm Grades?
• Changing to a midterm grade system reporting
specific grades (A, B, C, D, F with +/-) would
probably not improve retention.
– Simply maintaining a C or better is what we find to
be critical to retention.
• Participation in a voluntary system of midterm
grades would almost surely be lower than
early warning system.
• Mandatory midterm grades seem redundant
in an era of online gradebooks.
Low Hanging Fruit?
• Is there any easy change to the system that
would have the potential to extend the
reach, encourage more participation, etc.?
– Yes! Online courses
• Online courses had been excluded from the
population of courses for which early
warnings were submitted.
– Why? Possibly because when online courses
first appeared, they were often asynchronous,
not to mention experimental. Not so today.
Committee Recommendations
• Maintain the existing timetable for submission of early
warning grades
• Maintain the system of early warning grades, as opposed
to specific midterm grades
• Expand the early warning grade population to include
online courses
• Recommend to Provost that resources be allocated for the
purchase of GradesFirst for the Office of Academic
Services
• Study the effectiveness of the inclusion of online courses
and the use of GradesFirst after their implementation
Questions???
Fall 2012 IACRAO Conference
Session 25.25