Influential Interventions: Assessing Supplemental Instruction in Gateway STEM Courses
James Diedrick, PhD, Associate Dean of the College; Molly Smith, PhD, Director of the Science Center for Women
Agnes Scott College, Atlanta Georgia, USA
Background
Supplemental Instruction Implementation
Key Findings
Abstract
SI offered for 10 different courses in 4 departments
Positive impact of regular SI attendance
Chemistry 101/150 & 102/220 Introductory Sequence
SI offered FA11, SP12, FA12, SP13
Attendance Incentive Required for 5% of course grade
(attend 1 SI session or 1 hour at learning center weekly)
90%
80%
Math 117 Pre-Calculus
SI offered FA12
Attendance Incentive Exam bonus points
American Indian/Alaska Native
70%
Diversity of our entering class
has increased dramatically
over the past 20 years…
Unknown
60%
Hispanic
50%
Two or more races
40%
Asian/Pacific Islander
30%
Non-resident International
20%
African American
10%
White
0%
1992
2002
4.4%
5.4%
34.6%
8.5%
… yielding a diverse student
body of ~900 women.
Non-resident International
10.7%
Hispanic
•
•
Two or more races
Unknown
Asian/Pacific Islander
•
•
COMMON
All students encouraged to attend SI
SI sessions offered weekly, in the
evenings
SI sessions led by LAs
LAs attend course lectures regularly
33.1%
Motivation
For many years, science and mathematics faculty noticed that URM students were often less successful than
majority students in their introductory courses. In 2009, faculty and staff analyzed performance data from
the preceding 5 years to better document what had been isolated and anecdotal observations. Those data
confirmed faculty observations. Discussions about how to address the disparity in student performance led
to the current project, which implements, strengthens and assesses several academic support interventions.
Objective
Increase student success and persistence in STEM disciplines by adding peer-led supplemental instruction
(SI) sessions to gateway math and science courses
• All students are encouraged to attend SI sessions, with incentives in most classes
• Specific impact on URM and first generation students (“target” students) determined through
assessment
Percentage of Students
Physics 110 & 111 Introductory Sequence
SI offered FA12, SP13
Attendance Incentive Encouragement
Faculty had some flexibility in SI implementation
White
African American
improved
Biology 191 & 192 Introductory Sequence
SI offered FA11, SP12, FA12, SP13
Attendance Incentive Encouragement
Math 119 Calculus II
SI offered SP12, SP13
Attendance Incentive Exam bonus points
Ethnic Diversity of
Student Body 2012-13
60%
VARIED
• Method of promoting attendance
- Frequent encouragement to attend and mention of sessions
- Exam bonus points awarded (1 point/session attended)
- Requirement (1 session or 1 hour at learning center/week)
• Number of sessions per week
- 2-6 sessions per week, offered Sun-Thurs, depending on class
size & number of sections
• Level of guidance professors provided LAs for SI content
- Low: LAs develop nearly all content, based on their
observations during lecture & recollections from when they
took the course
- Mid: Profs & LAs meet weekly to discuss possible SI topics and
profs provide some guidance on specific exercises/problems
- High: Profs provide detailed worksheets and meet regularly
with LAs
Supplemental Instruction Participation Trends
Student participation increasing incrementally
• Approximately 80% of all students enrolled in courses offering SI participated in one or more workships
during the two-year period
improved
worsened
50%
40%
30%
20%
worsened
50%
40%
30%
20%
10%
10%
0%
0%
0 sessions attended
Math 118 Calculus I
SI offered FA11, SP12, FA12, SP13
Attendance Incentive Exam bonus points
3.3%
2012
60%
Chemistry 240 Organic I
SI offered SP13
Attendance Incentive Exam bonus points
Diversity on Campus:
Total First-Year Population
100%
Grade change from midterm to final for
non-target students
Grade change from midterm to final for
target students
Percentage of Students
In 2011, with support from a two-year Teagle Foundation grant (shared with Davidson College), Agnes Scott
College initiated a targeted effort to improve student persistence and success in STEM disciplines, especially
among our significant and growing population of first-generation and underrepresented minority students (in
2011-12, first-gen and URM students were the largest demographic within our first-year class, making up
nearly 45% of the student body). Adding peer-led supplemental instruction to gateway math and science
courses and hiring a Coordinator to enhance support offered through our Math and Science Learning Centers
have been high-impact changes, with positive results demonstrated by directed assessment of student
performance, focus groups and surveys.
6+ sessions attended
0 sessions attended
6+ sessions attended
More students who attended 6+ workshops in 2011-13 saw their grades increase, while more students
who attended 0 workshops saw their grades worsen (in both the target and non-target populations).
This trend is more pronounced for target students.
SI attendance correlates with higher final grade
Average Final Grade by SI or Non-SI
4.00
3.66
3.33
3.00
2.66
2.33
2.00
1.67
1.33
1.00
0.67
0.33
0.00
Non-SI = 0 or 1 Sessions attended, SI = 2+ Sessions attended
SI < non-SI
SI > non-SI
avg non-SI grade
In 14 out of the 19 classes in which SI
sessions were implemented, students
who participated in two or more
sessions earned a better average final
grade than students who attended one
session or none at all...
...this improvement was an average of 1 1/2 letter grades,
with a high in CHE-240 where SI students earned an average of B+ and non-SI students averaged a C-.
Institutional Changes
These positive impacts have resulted in faculty and administration support for continuing SI and expanded
resources for STEM courses in the following ways:
• SI will continue beyond the grant period with the
support of college funds
• Faculty are committed to adding SI to more courses
• The Resource Center for Math and Science will have
significant dedicated space within our newly renovated
Academic Resource Center (opening scheduled 2014)
• Coordinating the RCMS now officially falls within the
role of our Director of the Science Center for Women
Overall increase in participation,
2011-13 by course
Coordinator of Resource Center for Math & Science (RCMS)
• Assists in facilitating the implementation and assessment of the SI program
• Supports & coordinates the work of all LAs and peer tutors (individual peer tutors have been in place
for two decades)
• Addition of this new position has provided an opportunity to increase level & variety of academic
support available to our students in math and the sciences
Percentage of students who attended
at least one workshop
New Academic Support Roles
Non-Target 2011-12
Non-Target 2012-13
Target 2011-12
Target 2012-13
100%
80%
60%
40%
Recommendations
For target students, participation increased
in 5 of 6 classes by an average of 10%.
For non-target students, participation
increased in just 3 of 6 classes, but overall
participation was up over 6%...
20%
0%
50% Increase in Participation
per Student in 2nd Year of Offering SI
2011-12
Peer Learning Assistants (LAs)
• Sophomores, juniors or seniors selected by faculty; partner with individual faculty and course
• Training: 2-day session in August, ½ day session in January, and on-going training at regular meetings
throughout each semester
• Responsibilities: attending course lectures, leading SI sessions, developing workshop content (to
varying extents), holding 1:1 tutoring hours & attending regular staff meetings
• Grant support allowed for hiring students not eligible for traditional work-study
...and both target and non-target students
participated in more workshops on average,
showing that our SI program is
developing momentum.
2012-13
0
1
2
3
Non-target Students
4
5
6
7
Target Students
Average Number of Workshops Attended per Student
• A robust peer tutoring/mentoring system, coupled with SI, can help an institution shift from a “boot camp”
approach to gateway STEM courses to an approach that assumes and supports student success
• SI sessions are most effective when the LAs and the professor establish specific activities and learning goals
for each session and put these in writing.
- An excellent template for this can be found in Supplemental Instruction: Improving First-Year Student
Success in High-Risk Courses, by Martin C. Deanna and David R. Arendale (University of South Carolina)
- Flexibility in method and approach is important for both professors and LAs
• Including sophomores, juniors and seniors in the LA cohort:
- Maintains continuity when seniors graduate
- Makes first year students feel more comfortable approaching LAs
- Allows rising sophomore and junior LAs who previously attended SI sessions as students to build on and
share their own classroom experiences and knowledge
• Regular group meetings of the Resource Center Coordinator with the LAs and faculty are critical.
Discussions at these meetings led to the development of new resources and changes that improved
academic support for students
• Incentives or requirements for attendance are essential to the success of SI programs.
• To access this poster on the Agnes Scott College website, go to
http://www.agnesscott.edu/assessment/files/documents/TeagleSTEMposter.pdf