Page 1 of 13
ASSESSING EFFECTIVENESS OF PEER ASSISTANCE PROGRAMMING
Sally Lipsky, Ph.D., Peer Assistance Coordinator ([email protected])
Center for Learning Enhancement, Developmental Studies Department
Indiana University of Pennsylvania, Indiana, PA 15705 U.S.A.
www.iup.edu/devstudies
At Indiana University of Pennsylvania peer assistance consists of Supplemental Instruction and Walk-In
Tutoring. Both target high-risk freshman-level courses, that is, courses with “D/F/W” rates of 30% and higher.
Effectiveness of these academic support models is measured accordingly:
I. Student Participants (undergraduate students using the programs ). Outcomes are reported for participation
rates, final course grades, success rates as measured by % of D/F/W’s, as well as annual satisfaction surveys
sent via e-mail.
II. Peer Educators (undergraduate, paraprofessionals leading the sessions ). Outcomes are measured according
to objectives set forth in the 1-credit training course, which all must successfully complete before starting their
paid positions. Also, outcomes are reported for end-of-term surveys and focus groups, which produce
qualitative information regarding the academic, personal/social, and career effect of peer educators’ work
experiences.
III. The Institution (Indiana University of Pennsylvania). Outcomes are reported according to student persistence
and revenue saved by the peer assistance programs. Since students’ participation results in an approximate
10% higher persistence rate, peer assistance is quite cost-effective for the institution.
Lipsky, S. A. Assessing Effectiveness of Peer Assistance Programming. 20th International Conference on The First-Year
Experience, Kona, Hawaii, July 2007.
Page 2 of 13
Student Participants:
Participants Annual Satisfaction Survey
SUPPLEMENTAL INSTRUCTION
WALK-IN PEER ASSISTANCE
* E-mailed to 347 students attending 3 or more
times; 109 responded — an overall response
rate of 31%.
* E-mailed to 181 students attending 2 or more
times; 27 responded — an overall response rate
of 15%.
My participation helped me learn course
content.
A. Yes
75%
C. No
3%
B. Somewhat 22%
D. Not sure0%
My participation helped me to develop
effective learning strategies & study skills.
A. Yes
46%
C. No
6%
B. Somewhat 48%
D. Not sure1%
The leader was prepared and organized.
A. Yes
84%
C. No
3%
B. Somewhat 13%
D. Not sure0%
The leader incorporated active
participation by all students.
A. Yes
79%
C. No
3%
B. Somewhat 17%
D. Not sure 1%
Most valuable features of SI?
– Worksheets, practice exams, study
guides.
– Increased understanding of course
material.
Suggestions for improving?
– More time, sessions, individual help.
My participation helped me complete
assignments & learn course content.
A. Yes
85%
C. No
4%
B. Somewhat 11%
D. Not sure0%
My participation helped me develop more
effective learning strategies & study skills.
A. Yes
70%
C. No
4%
B. Somewhat 26%
D. Not sure0%
The leader responded to my questions &
needs.
A. Yes
96%
C. No
0%
B. Somewhat 4%
D. Not sure 0%
Most valuable features?
– Helpful peer educators.
– Flexible, able to come and go as
needed.
Suggestions do you have for improving?
– More tutors, more times.
* Results for fall term 2006.
Lipsky, S. A. Assessing Effectiveness of Peer Assistance Programming. 20th International Conference on The First-Year
Experience, Kona, Hawaii, July 2007.
Student Participants:
Participants Participation Rates, Final Course Grades, and Success
Rates
Supplemental Instruction
Offered in 29 high-risk* natural science
course sections for 2006-7.
Walk-In Assistance
Offered for lower-level, high-risk* courses
with many sections.
 Mean final course grade for participating
students (1 or more times) was 2.08 vs. 1.55
for non-participants.
 Approximately 16% of students in targeted
courses participated.
 This .53 mean grade difference exceeds the
national mean of .29 for 4-yr. public
universities.
 Participants’ mean final course grade was
1.96 vs. 1.69 for non-participants, a
difference of .27.
 Participants averaged 7 sessions per term.
 Participants averaged 3 sessions per term.
 Participants’ mean D/F/W rate* was 32% vs.
48% for non-participants, a difference of
16%.
 Participants’ mean D/F/W rate* was 34% vs.
47% for non-participants, a difference of
13%.
 Total contact hours = 6,093.
 Total contact hours = 1,887. (1,265 fall;
622 spring)
 Average contacts per week = 218.
Results reported for academic year 20067.
 Average contacts per week = 93 fall; 48
spring.
Reporting Outcomes for Student Participants
PARTICIPATION RATES
 Participation rate for students in courses with peer assistance. (Contrast with national data for 2-yr./ 4-yr.
institutions.)
 Mean number of sessions participants attended (refer to results of participation study).
 Total student contacts for peer assistance at your institution. Can you chart growth rates?
MEAN FINAL GRADES
 Difference in mean final course grades: participants vs. non-participants.
 Are participants’ mean final course grades at or above the 2.0 cut-off for good academic standing?
 Note criteria for identifying participants: students attending 1 or more times during the term, 3 or more times, or
other.
 Contrast your mean to national mean for 2-yr./ 4-yr. private or public institutions.
D/F/W RATES* As an academic retention model, peer assistance targets high-risk,
high-risk introductory-level courses—
courses in which 30% or more students earn D’s, F’s, or withdraw. High D/F/W rates increase costs for students
and the institution since students repeat courses (filling valuable seats), become increasingly frustrated and less
satisfied, and are more likely to leave. As a result, the institution expends more money to recruit new students as
replacements. Peer assistance intervenes in this costly cycle via peer-led sessions focusing on how to learn course
content. Thus, a measure of success of peer assistance is a reduction in D/F/W’s for historically difficult courses.
 D/F/W rate for participants vs. non-participants.
 Overall rate for peer assistance courses. Has it fallen below the 30% standard for high-risk courses?
Student Participants:
Participants Incoming Characteristics
One common assumption is that students who choose to participate in SI are academically
stronger than students choosing not to participate. However, previous research has
demonstrated that this is not the case—both SI participants and non-participants tend to
possess similar incoming characteristics.
For the 2006 admit year, I examined mean SAT scores for beginning freshmen in the same
major and college—Nursing/Allied Health—at IUP.
Hypothesis: There is no difference in academic ability, as measured by SAT total scores,
between freshmen choosing to participate in SI and freshmen choosing not to participate in
SI for Biol 105: Cell Biology.
Results: No statistical difference in SAT scores was revealed between the two groups.
However, mean SAT scores for freshmen choosing not to participate in SI was 24 points
higher than for the participants.
NO, did not
YES, attended SI *
attend SI for Biol
for Biol 105
105
n = 83
n = 64
Mean SAT total score
998
974
difference mean SAT
24 favoring non-participants
scores
t-value (2-tailed)
.43 – not significant at .05 level
Mean final grade Biol
1.43
1.88
105
difference mean final
.45 favoring SI participants
grades
FALL 2006
Freshman Nursing &
Allied Health majors
*Averaged 8.6 sessions during fall term.
Note: The freshmen attending SI had lower entering SAT scores yet earned higher final
grades in Cell Biology, underscoring the positive affect of peer-led SI sessions, especially
when students participate regularly as did these freshman participants.
Peer Educators: Outcomes measured according to Objectives for 1-credit
Training Course
ROLE OF THE PEER EDUCATOR: THEORY, PRACTICE, & ASSESSMENT
COURSE OBJECTIVES:
OBJECTIVES Students will be able to:
 Describe purposes & impacts of peer educator roles.
 Observe & summarize characteristics of effective peer-led sessions.
 Understand elements of facilitative helping & collaborative learning.
 Demonstrate effective problem solving skills.
 Demonstrate knowledge of appropriate resources for referring peers.
 Demonstrate understanding of how to apply college learning strategies.
 Assess performances of peer educators.
 Relate effective practices to student success outcomes.
EVALUATION METHODS
 Actively participate in class activities (discussions, simulated sessions, role-playing) & complete all
assignments. 25% of final grade
 Observe & critique four peer-led sessions. 50% of grade
 Lead & critique a trial session. 25% of grade
Peer Educators: Results of End-of-Year Surveys
Peer educators report benefits of their work experience:
Academic Growth
 100% reported the position influenced their desire to learn & decision-making skills.
 91% reported improved ability to think through complex issues & solve problems.
Social Growth
 91% reported improved ability to work effectively & cooperatively with others.
 83% developed relationships with individuals of differing backgrounds.
 75% indicated that their work helped them accept the uniqueness of others.
“I have a special way to present the topic of aerobic respiration to students. I can see the
light bulbs switching on, and it amazes me that I can have that kind of power and effect!”
Personal/Career Growth
 75% indicated that their work helped them to define a career path for themselves.
“I have decided to get my masters degree in biology so that I can teach on the college
level. My experiences as a peer educator have led me to this decision.”
 81% rated their role as having a substantial impact in their satisfaction with IUP.
“I love the SI program. My job has helped me to become a part of the campus community,
and I learn from the students I work with. I can’t imagine life without it!”
Peer Educators:
Educators Qualitative Study
TITLE: Effects of Work Experiences for Undergraduate Peer Educators
PURPOSE: Assess the effects of the work experience for peer educators.
METHOD:
 Thirteen peer educators participated; all worked at least two terms.
 Each participated in 1 of 3 focus group sessions.
 A graduate student facilitator presented open-ended questions
experiences & provided prompts, as needed, to spark discussion.
about
work
 Each discussion was taped and transcribed.
 Two investigators collated and categorized responses independently.
PRELIMINARY RESULTS: Subjects consistently reported benefits of their work experience in terms of increased:
increased
 Academic skills.
 Communication & relationship-building skills .
 Personal & professional skills.
TENTATIVE CONCLUSION: Peer educators regard their positions as more than just another campus job. Besides
monetary compensation, they gain cognitive, social, personal, and professional growth and rewards.
Institution:
Institution Measuring Cost-effectiveness of Peer Assistance for Freshman
Students
Approximately 10% more students persist until graduation than do non-participating
students with similar characteristics.*
Cost-Effectiveness per Term
1. Approximately 200 freshmen attend at least 1 SI session for a targeted course during a term at
Indiana University of PA (IUP).
2. If 10% more persist,* approx. 20 more freshmen remain at IUP as a result of SI participation.
3. A full-time undergraduate student residing in Pennsylvania contributes approx. $2,900 a term for
tuition & fees.
4. The state contributes about $2,500 per student per term.
5. Total revenue generated per student is about $5,400 per term.
6. If 20 more students remain at IUP due to participation in SI, the approx. retained revenue
per term is $108,000 – & every term after until each of the 20 students graduates.
* Congos, Dennis (2001). How SI Retains Revenue for Colleges and Universities. University of
Central Florida.
Institution:
Institution Measuring Cost-Effectiveness over Four Years
Cost-Effectiveness over Four Years
1. The next term another 200 freshmen attend at least one SI session, resulting in another 20
students remaining at the institution.
2. Another $108,000 in tuition & fees is retained the next term, or approx. $216,000 is retained over
two terms.
3. Each term SI generates a new cadre of freshman participants.
4. For each group of freshmen attending SI, the institution retains approximately $756,000 in
revenue through graduation of the students.
These figures do not include:
 Costs to recruit students to replace 10% otherwise lost to attrition.
 Added revenue from upper-class students attending SI.
 Costs of peer assistance programming (personnel, materials, facilities).
Based on an average of 200 FRESHMAN participants per term, resulting in 20 students persisting each term until graduation. Costs spent
on recruiting students to replace the 10% of students otherwise lost to attrition and revenue from participating upper-class students are not
included. Costs of peer assistance programming (personnel, materials, facilities) are not deducted.
YEAR 1:
YEAR 1:
YEAR 2:
YEAR 2:
YEAR 3:
YEAR 3:
YEAR 4:
YEAR 4: End
TOTAL
End of
End of
End of FALL
End of
End of FALL
End of
End of FALL
of SPRING
RETAINED
FALL
SPRING
TERM
SPRING
TERM
SPRING
TERM
TERM
REVENUE
TERM
TERM
TERM
TERM
Group A
Group A
Group A
Group A
Group A
Group A
Group A
Group A
Group A
20
20
20
20
20
20
20
20 students
7 terms of
Freshme
students
students
students
students
students
students
saved an
persistenc
n saved
saved
saved an
saved an
saved an
saved an
saved an
added
e saved
0.
$108,000.
added
added
added
added
added
$108,000.
$756,000.
$108,000.
$108,000.
$108,000.
$108,000.
$108,000.
Group B
Group B
Group B
Group B
Group B
Group B
Group B
Group B
20
20
20
20
20
20
20 students
6 terms of
Freshmen
students
students
students
students
students
saved an
persistenc
saved 0.
saved
saved an
saved an
saved an
saved an
added
e saved
$108,000.
added
added
added
added
$108,000.
$648,000.
$108,000.
$108,000.
$108,000.
$108,000.
Group C
Group C
Group C
Group C
Group C
Group C
Group C
20
20
20
20
20
20 students
5 terms of
Freshmen
students
stu
stu
students
saved an
persistenc
saved 0.
saved
den
den
saved an
added
e saved
$108,000.
ts
ts
added
$108,000.
$540,000.
sav
sav
$108,000.
ed
ed
an
an
add
add
ed
ed
$10
$10
8,0
8,0
00.
00.
Group D
Group D
Group D
Group D
Group D
Group D
20
20
20
20
20 students
4 terms of
Freshmen
students
students
students
saved an
persistenc
saved 0.
saved
saved an
saved an
added
e saved
$108,000.
added
added
$108,000.
$432,000.
$108,000.
$108,000.
Group E
20
Freshmen
saved 0.
Group E
20
students
saved
$108,000.
Group F
20
Freshmen
saved 0.
$0
retained
Fall
YEAR 1
$108,000
retained
Spring
YEAR 1
$216,000
retained
Fall YEAR 2
$324,000
retained
Spring
YEAR 2
$432,000
retained
Fall YEAR 3
$540,000
retained
Spring
YEAR 3
Group E
20
students
saved an
added
$108,000.
Group F
20
students
saved
$108,000.
Group G
20
Freshmen
saved 0.
$648,000
retained
Fall YEAR 4
Group E
20 students
saved an
added
$108,000.
Group E
3 terms of
persistenc
e saved
$324,000.
Group F
20 students
saved an
added
$108,000.
Group G
20 students
saved
$108,000.
Group F
2 terms of
persistenc
e saved
$216,000.
Group G
1 term of
persistenc
e saved
$108,000.
Group H
0 terms of
saved $0.
$3,024,000
Total
retained
after 4
years
Group H
20 students
saved 0.
$756,000
retained
Spring YEAR
4