Influential Interventions: Improving STEM
Learning Outcomes for Underrepresented Students
James Diedrick, Associate Dean of the College
Drew Homa, Academic Assessment Coordinator
25th International Conference on The First-Year Experience
Vancouver, B. C. • July 17, 2012
Agnes Scott College
Atlanta, Georgia
Student Body as of Fall 2011:
883 students, representing 41
states and territories and 29
countries
89% of traditional students live on
campus
11% are international students
More than 40% are students of
color.
About 40% will study abroad
before they graduate
Historically and presently, Agnes
Scott students have earned
academia’s most prestigious
scholarships including the Marshall,
Rhodes, Fulbright, Goldwater, the
Pickering Fellowship and the Gates
Millennium Scholarship.
Students in the Generating Excellence in Math & Science
Summer Scholars Program (GEMS), Summer 2011
Our Commitment to STEM Education
Majors: Biology, Biochemistry and Molecular
Biology, Chemistry, Mathematics, MathematicsPhysics, Mathematics-Economics, Neuroscience,
Public Health, Physics & Astronomy
Dual degree programs in Engineering,
Computer Science and Nursing with Emory
University & Georgia Tech
State-of-the art facilities in new science
building, observatory on campus
Emphasis on supporting & encouraging women
in pursuit of STEM fields
Increasing Ethnic Diversity
1992-2011
1992
2011
2002
17.9%
32.5%
4.0%
3.9%
74.2%
White
40.4%
44.1%
4.5%
5.2%
57.9%
Non-resident International White
4.7% 10.8%
Increasing Ethnic Diversity
1992-2011
100%
Diversity on Campus:
Total
Undergraduate
Population
90%
80%
70%
100%
Diversity on Campus:
Total
First-Year
Population
90%
80%
70%
60%
American Indian/Alaska Native
60%
American Indian/Alaska Native
50%
Unknown
50%
Unknown
Hispanic
40%
30%
20%
Two or more races
Asian/Pacific Islander
Asian/Pacific Islander
Non-resident International
20%
Non-resident International
White
0%
Two or more races
30%
African American
10%
Hispanic
40%
African American
10%
White
0%
1992
2002
2011
1992
2002
2011
Achievement Gap for URM STEM Students

Analysis of performance in introductory STEM
courses clearly demonstrated a trend that faculty
had long noticed (data from 2004-2009)

Achievement gap persists even for students who
choose to major in a STEM discipline
Percentage of STEM Majors Having
a GPA ≥3.33 (2010)
GEMS Summer Scholars on field trip
to Georgia Aquarium, 2011
 Began 2007
 Year-round program aimed at supporting the
success of students pursuing majors in mathematics
and/or the natural and physical sciences
 GEMS Summer Scholar Program, 2009-2011  Activities include bi-monthly meetings, excursions,
service projects and collaboration with ASC faculty
 Students encouraged to take advantage of
academic support centers.
 Living-Learning component initiated for 2012-13
year.
Generating Excellence
in Math & Science
GEMS Program
Factors influencing success
& persistence in STEM
Family
Background
Skills &
Abilities
Classroom Climate
Stereotype Threat
Prior
Schooling
Feedback on Learning
Inquiry-Based Learning
Decision to
enroll in STEM
class/major
Exposure to RealWorld Applications &
Careers
Extracurricular
Activities
University of Michigan Center
for Research and Learning
Peer Group Interactions
Potential for peer-led
Supplemental Instruction in
gateway math & science
courses to have a positive
influence
Academic
Integration
Re-evaluate
commitments
& goals
Social
Integration
Decision to
persist or
depart from
STEM
Teagle Foundation Support for
Interventions to Promote STEM Student Success
 Two-year grant received in collaboration with Davidson College to develop
academic support initiatives that particularly benefit first generation and
URM students
 Initiative runs from August 2011-May 2013
 Goal includes use of student learning outcome data to continue the levels
of academic support beyond the funding period
 Initiative includes joint meetings of two institutions, workshops on campus
climate issues, stereotype threat, STEM pedagogy
Project Objectives
 Increase student success and persistence in STEM disciplines by adding
peer-led supplemental instruction (SI) sessions to gateway math and
science courses
 All students are encouraged to attend SI sessions
 Specific impact to URM and first generation students determined through
assessment
 SI has been associated with more dramatic gains among URM students than
among their peers*
 Addressing/improving campus/classroom climate issues (Diverse
Learning Environments Survey (DLE) administered Spring 2012,
comparative data to be released September 2012)
* Rath, Kenneth, et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance
and Retention of URM Students” (CBE – Life Science Education 6 [2007]:203-216).
New forms of academic support
Coordinator for Resource Center for Math & Science (RCMS)
 Assisting in facilitating the implementation and assessment of the SI program
 Supporting & coordinating the work of all LA’s and peer tutors (individual peer tutors have
been in place for two decades)
 Increasing the level & variety of academic support available to our students in math and the
sciences
Peer Learning Assistants (LA’s)
 Juniors or seniors selected by faculty
 Training: 2-day session in August, ½ day session in January, and on-going training at regular
meetings throughout each semester
 Responsibilities: attending course lectures, leading SI sessions, developing workshop content
(to varying extents), holding 1:1 tutoring hours & attending regular staff meetings
 Grant support allowed for hiring students not eligible for traditional work-study
Math & Science Learning Center Coordinator
Dr. Molly Smith, Math & Science Learning
Center Coordinator
Ph.D. in Chemical Engineering from University
of Michigan
6 years experience working in a university
environment as a graduate student, including
voluntary service roles in coordination,
leadership and planning of education programs
and events
5 years corporate experience managing
projects and programs involving team
members in multiple locations and partners at
other companies and universities; included
teaching intensive mini-courses on biology and
chemistry fundamentals for employees
Dr. Molly Smith
Ethnic & Racial Diversity of LA’s
2011-12
2012-13
9%
17%
8%
36%
50%
55%
25%
White
White
Total of 11 LA’s for 2011-12 and 12 LA’s for 2012-13
SI Implementation: Overview
Implemented SI in 6 gateway math & science courses
(13 total sections, 194 individual students) during 2011-2012
FALL 2011
Biology 191: Cell and Animal
Biology
Chemistry 101: Fundamental
Concepts of Matter and
Reactions
Math 118: Calculus I
SPRING 2012
Biology 192: Molecular Biology
and Genetics
Chemistry 102: Periodicity and
Chemical Reactions
Math 118: Calculus I
Math 119: Calculus II
SI Implementation: Overview
Some aspects of implementation were common across courses, while others
varied by course/professor
COMMON
 All students encouraged to
attend SI
 SI sessions offered weekly, in the
evenings
 SI sessions led by peer Learning
Assistants (LA’s)
 LA’s attend course lectures as
often as possible
VARIED
 Method of encouraging
attendance
 Number of sessions per week
 Level of guidance professors
provided LA’s for SI content
Supplemental Instruction Rubric
Session Date & Day of Week: __________________ SI Leader: ______________
Course: ____________________ Course Instructor: _______________________
Objective: What are the two most difficult concepts the students need to work
on today?
_________________________________________________________________
_________________________________________________________________
Beginning reminders:
Arrange seats in a circle
Hand out Participation Log
Set agenda with group
Remember to relax and be flexible!
Supplemental Instruction Rubric - continued
Content to cover
Processes to use*
Possible Processes: Informal Quiz, Matrix, Reciprocal Questioning,
Paired Problem-Solving, Turn to Use Your Partner, Note Processing,
Problem-Solving Rubric, Formal Definition (or ID’s), Text Review
(Divide and Conquer), Pictorial Representations, Sequencing
Closure options: Predict next lecture, summarize session, informal
quiz, 1-minute writing
--Rubric adapted from Deanna, Martin C. and David R. Arendale, Supplemental Instruction:
Improving First-Year Student Success in High-Risk Courses. (National Resource Center for the
Freshman Year Experience, University of South Carolina, 1992).
SI Implementation: Details Fall 2011
LA’s
SI/week
Attendance
Incentive
3 sections
3 profs
4
4
Encourage
mid
76
3 sections
2 profs
2
4
Require
high
29
1 section
1 prof
2
Exam Bonus
Points
(starting½ way)
high
Course
Enrolled
BIO 191
95
CHEM 101
Math 118
# Sections &
Profs
2
Guidance
to LAs*
Definitions for level of guidance provided to LA’s
Low: LA’s develop nearly all content, based on their observations during lecture & recollections from when they
took the course
Mid: Profs & LA’s meet weekly to discuss possible SI topics and profs provide some guidance on specific
exercises/problems to do
High: Profs provide detailed worksheets and meet regularly with LA’s
SI Implementation: Details Spring 2012
Course
BIO 192
CHEM 102
Enrolled
# Sections &
Profs
63
2 sections
2 profs
60
LA’s
SI/Week
Attendance
Incentive
3
3
Encourage
Low
Encourage*
Mid*
Require*
High*
2 sections
2 profs
2
Guidance
to LAs
4
MATH 118
30
1 section
1 prof
2
2
Exam bonus
points
High
Math 119
29
1 section
1 prof
1
2
Exam bonus
points
High
* Each CHEM 102 professor used a different approach to encourage attendance
Assessment Methodologies
Quantitative Data
SI session attendance logs
Mid-term and Final grades
Descriptive student
variables
(class year, URM, FGEN, etc.)
Qualitative Data
Focus group interviews with
First-Year students in STEM
gateway courses
Meetings and focus groups
with Learning Assistants
Workshops and discussions
with STEM faculty using LA’s
and SI
Session Attendance Trends
 194 Individual students participated
 Of those, 135 (70%) were “Target” students, either FGEN or URM
 Since some students took more than one course, we have a total
of 346 class-students who we studied.
 Of those 346 students, 254 (73%) of them participated in at least 1
workshop
 The ratio of Target students who participated in workshops is the
same as those attending the class (108 of 156 = 70%.)
SI Attendees Perform Better Overall
4
Average 3Final Grade
SI Group
Non-SI
Group
2
1
0
191
BIO
192
101
CHEM
SI Group: attended 2 or more sessions
102
118 FA 118 SP 119
MATH
Non-SI Group: attended 0 or 1 session(s)
Varied Levels of Session Attendance
Times Attended
Semester
11/FA
11/FA
11/FA
11/FA
11/FA
11/FA
11/FA
12/SP
12/SP
12/SP
12/SP
12/SP
12/SP
Class
BIO-191
BIO-191
BIO-191
CHE-101
CHE-101
CHE-101
MAT-118
BIO-192
BIO-192
CHE-102
CHE-102
MAT-118
MAT-119
# students
28
41
17
24
27
23
26
21
33
23
37
25
24
0
46%
22%
41%
8%
37%
17%
31%
33%
12%
30%
41%
24%
4%
1-2
11%
24%
29%
8%
15%
22%
42%
19%
15%
13%
19%
16%
25%
3-5
32%
20%
6%
33%
30%
22%
19%
14%
30%
17%
16%
16%
33%
6+
11%
34%
24%
50%
19%
39%
8%
33%
42%
35%
22%
44%
38%
Positive Impact of Regular SI Attendance
From Mid-term Grade to Final Grade...
Grade
Improved or was an A at mid-term
No Change
Worsened
Target Students
(40%) 95
(22%) 53
(38%) 89
Non-Target Students
(55%) 55
(43%) 43
(16%) 16
...Based on Number of Sessions Attended
0.5
0.4
0.3
Percentage of Students
0.5
# times
attended
0.4
0
0.2
0.3
Percentage of Students
0
0.2
0.1
0.1
0
0
I mproved or A at mid-term
# times
attended
I mproved or A at mid-term
Comparison of Fall and Spring Performance
From Mid-term Grade to Final Grade
From Fall 2011 to Spring 2012
Of the 102 students who took an SI class in both semesters:
More sessions attended in
Spring than Fall
Same sessions attended in
Spring as in Fall
Fewer sessions attended
in Spring than Fall
No sessions attended Fall
or Spring
More
improvement
in Spring than
Fall
Same
improvement
in Spring and
Fall
19
6
14
2
4
5
2
2
10
3
21
3
11
Less
Earned A's at
improvement Mid-term and
in Spring than Final, both Fall
Fall
and Spring
Samples of Student Feedback—first years
[My LA] helps me to restore my faith in chemistry and want to study
more. She has been very helpful, kind, patient, and honest. She
explains topics in real world scenarios by making references to
reality TV shows, elementary schools, and everyday life situations.
At times when the textbook and word problems become to wordy she
breaks them down into smaller simpler pieces that help me to piece
things together and think through the question in a more useful,
sensible way instead of getting stressed out and giving up. She
teaches us with authority while showing humility and compassion,
which makes it easier to relate to her and ask her questions that
might seem trivial. I do not know what I would have done this
semester without her help.
The [LAs] were complete life savers- always so friendly and able to
help. I absolutely loved them and was privileged to work with them.
Samples of LA Feedback
“What were some of your favorite things about being a Teagle
Learning Assistant?”
 Getting to work with students and feeling a sense of
accomplishment knowing I helped them learn something.
 Students trusted and respected me and I had an active role in
helping them succeed.
 Helping others understand concepts and seeing them apply them
to all areas.
 Having a close bonding with first year calculus students
 Having the experience of teaching college material.
 Having fun creating new forms of teaching and activities
 Sharing the excitement of every “aha” moment; and of course
introducing the world’s coolest subject to a new generation
Next Steps/Future Plans…
 SI Rubric used by all faculty
 Move from encourage to require attendance at SI sessions
 Facebook page to facilitate contact between students and LAs
 Sharing positive impacts we’ve seen with students
 Analyzing DLE data and comparing ASC with national data
 Adding additional classes to SI/LA initiative (Physics)
 Working with Davidson College to compare findings and best
practices
Selected References
Brown, Marie Kendall, et al. “Teaching for Retention in Science, Engineering and Math Disciplines: A
Guide for Faculty” University of Michigan Center for Research and Learning Occasional Paper No. 25,
2009.
Davis, Jeff. The First-Generation Student Experience: Who They Are, Their Characteristics, and
Strategies for Improving Their Persistence and Success. Stirling, Virginia: Stylus, 2010.
Deanna, Martin C. and David R. Arendale. Supplemental Instruction: Improving First-Year Student
Success in High-Risk Courses. The Freshman Year Experience: Monograph Series Number 7. 3 rd Ed.
Columbia, South Carolina: National Resource Center for the Freshman Year Experience, 1992.
Eagan, Kevin & Jessica Sharkness. “A Strong Start in the Sciences: Factors Influencing Minority Students’
Academic and Social Engagement.” Conference Presentation, 28 th Annual Conference on the First Year
Experience, Orlando, FL February 2009.
Freeman, Scott, et. al. “Prescribed Active Learning Increases Performance in Introductory Biology. CBE—
Life Sciences Education, 6 (2007): 132–139.
Jones Taylor, Valerie and Gregory M. Walton. “Stereotype Threat Undermines Academic Learning
(Personality and Social Psychology Bulletin 37 (2011): 1055–1067.
Rath, Kenneth A., et. al. “Supplemental Instruction in Introductory Biology I: Enhancing the Performance
and Retention of Underrepresented Minority Students.” CBE—Life Sciences Education 6 (2007): 203–
216.
Steele, Claude. “A Threat in the Air: How Stereotypes Shape Intellectual Identity and Performance”
American Psychologist 52 (1997): 613-629.
Thanks to:
 Dr. Molly Smith, Math & Science Learning Center Coordinator
 Dr. Lerita Coleman Brown, Director of the Science Center for
Women
 Jennifer Cannady, Assistant
Dean of the College,
Director of Academic
Advising & Student Success
 Agnes Scott students, staff
& STEM faculty
 Agnes Scott ITS Staff
Bradley Observatory
And also thanks to:
 Dr. Laura Palucki-Blake, Assistant
Director of the Cooperative
Institutional Research Program (CIRP)
at the Higher Education Research
Institute, UCLA
 Dr. Valerie Jones Taylor, Assistant
Professor of Psychology at Spelman
College
 The National Center for
the First-Year Experience
and Students in Transition
 The Teagle Foundation
Dr. Laura Palucki Blake