International Journal for the Scholarship of
Teaching and Learning
Volume 2 | Number 1
1-2008
Overcoming Student Resistance to a Teaching
Innovation
Wendy Keeney-Kennicutt
Texas A&M University, [email protected]
Adalet Baris Gunersel
Texas A&M University, [email protected]
Nancy Simpson
College Station, Texas, USA, [email protected]
Recommended Citation
Keeney-Kennicutt, Wendy; Baris Gunersel, Adalet; and Simpson, Nancy (2008) "Overcoming Student Resistance to a Teaching
Innovation," International Journal for the Scholarship of Teaching and Learning: Vol. 2: No. 1, Article 5.
Available at: https://doi.org/10.20429/ijsotl.2008.020105
Article 5
Overcoming Student Resistance to a Teaching Innovation
Abstract
This mixed-methods study investigated student perceptions of an innovative educational tool and the
instructor strategies that helped change initial student resistance into acceptance and engagement. The
educational tool in this study is Calibrated Peer Review (CPR)™, a web-based program that uses writing as a
learning and assessment tool. Evaluations of CPR were analyzed from students in a general chemistry course
over seven semesters involving 1515 students. Analysis revealed reasons for students’ like or dislike of CPR
and how the instructor modified implementation to provide students a more positive experience. Analysis of
student perceptions suggests that successful implementation of new tools requires attention to potential
sources of student resistance at the outset as well as active listening and response to student concerns.
Keywords
Calibrated peer review, Educational tools
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Overcoming Student Resistance to a Teaching Innovation
Wendy Keeney-Kennicutt
Texas A&M University College
Station, Texas, USA
[email protected]
Adalet Baris Gunersel
Texas A&M University
College Station, Texas, USA
[email protected]
Nancy Simpson Texas
A&M University College
Station, Texas, USA n-
[email protected]
Abstract
This mixed-methods study investigated student perceptions of an innovative educational
tool and the instructor strategies that helped change initial student resistance into
acceptance and engagement. The educational tool in this study is Calibrated Peer Review
(CPR)™, a web-based program that uses writing as a learning and assessment tool.
Evaluations of CPR were analyzed from students in a general chemistry course over seven
semesters involving 1515 students. Analysis revealed reasons for students’ like or dislike of
CPR and how the instructor modified implementation to provide students a more positive
experience. Analysis of student perceptions suggests that successful implementation of new
tools requires attention to potential sources of student resistance at the outset as well as
active listening and response to student concerns.
Introduction
The transformation of college courses from teacher-centered to learner-centered often
involves the introduction of new methods of learning or assessment in which students play
an active, responsible role. Research suggests that students who have come to expect a
more passive role in their formal education may initially resist such new methods (e.g.,
Smith, Cooper, & Lancaster, 2002; Van Patten, 2000; Vuorela & Nummenmaa, 2004). As
Boud (1981) pointed out, “Student reticence and resistance to taking responsibility for
learning are likely to be among the first problems the teacher will meet” (p. 13). Often
technology-based innovative tools are implemented to enable more self-directed learning.
Diffusion theories focusing on the manner in which an innovation is accepted and adopted
by a group have been used to increase the adoption of various technologies in education
(Surry & Farquhar, 1997). One of the most widely-used diffusion theories is the Theory of
Perceived Attributes (Rogers, 2005) stating that an innovation has five attributes which
form the basis of judgment by the receiving group: (a) trialability (can be tried before
implementation); (b) observability (presents observable results); (c) relative advantage (is
relative to what is being used); (d) complexity (is not too complex); and (e) compatibility
(is compatible with practices and values). Researchers suggest that student attitudes and
expectations, which are related to practices and values, impact performance (e.g., Cheung
& Huang, 2005; Cuban, 1993). Students may resist innovative tools that prompt self-
https://doi.org/10.20429/ijsotl.2008.020105
1
Overcoming Student Resistance to a Teaching Innovation
directed learning since it requires a shift in student thinking about responsibilities of
learners and teachers (Akerlind & Trevitt, 1999). In addition to this, innovation—especially
when it is being imposed externally—may produce anxiety and disorientation (Akerlind &
Trevitt, 1999). Many experienced faculty, like other professionals, deal with problems like
this intuitively by reflecting on these student issues and changing teaching strategies to
meet their students' needs. This activity is described as reflection-in-action, which is the
process that allows one to restructure a project while working on it. The process is more
than "trial-and-error;" it is a collection of purposeful decisions that are made by a
professional after being surprised by an unexpected turn-of-events in the midst of the
project. The astonishment causes one to rethink and question an original premise. When an
original model of thought does not work, upon reflection one then arrives at new ways to
address the issue (Schön, 1983).
This paper presents a study exploring student response to a particular technological
innovation—Calibrated Peer Review (CPR)™—in an introductory chemistry class. Our
research team and coauthors of this paper consisted of the instructor of the course and two
faculty developers, who had been involved with helping science and mathematics faculty
design and implement CPR assignments. The instructor had used CPR for seven semesters
and had surveyed student perceptions at the end of each semester. She came to the faculty
developers with the survey results; while she had used student feedback informally to
improve her implementation, she was now interested in doing more in-depth analysis of the
data. Together, we approached the data with two research questions: (1) What do students
think about CPR as a learning tool? (2) What do student comments reveal about the reasons
for their acceptance of or resistance to CPR? The data included both quantitative information
from Likert-scale items and qualitative information from an open-ended question. As we
read comments from the early semesters of implementation, the faculty developers on the
research team were struck by fact that the instructor had persevered despite the early
vehemence of students’ resistance. This gave rise to a third research question: (3) Why and
how did the instructor persist? We believed that insights gained from this analysis would
enable us to help other faculty in their implementation of teaching innovations. To
investigate this question, we interrupted our analysis of student comments on several
occasions so that the faculty developers could confer with the instructor on how she
introduced CPR to the students, how she modified her approach, and why she persisted
even though students resisted. The instructor’s reflections enriched our combined
understanding of what had occurred while the student comments prompted probing
questions and deeper reflection. While the inquiry and analysis process was spiral-like
rather than linear, in this retrospective description we attempt to communicate the methods
and results in a linear fashion.
We begin with an explanation of CPR, which is followed by the study's methods, including
the instructor's description of the changes she made in class throughout the semesters (in
her own voice). Next, we discuss the findings of the quantitative and qualitative data,
which include the faculty member's reflections on her instructional modifications that
resulted in greater student acceptance of this innovation.
Calibrated Peer Review™
Calibrated Peer Review (CPR)™ is a web-based program that facilitates the use of writing as
a learning and assessment tool. Initially developed at UCLA for the Molecular Science
Project (http://cpr.molsci.ucla.edu/), one of the NSF-supported Chemistry Systematic
Reform Initiatives, CPR has been used in a wide range of disciplines. After instructors create
https://doi.org/10.20429/ijsotl.2008.020105
2
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
assignments using the authoring tools, students complete the assignments through three
phases: (1) Following the instructions, they access suggested resources, and write and
submit their essays; (2) they practice reviewing by grading three sample essays, created by
the instructor, which exemplify a low-quality, medium-quality, and high-quality essay
(called calibration essays); and (3) they grade the essays by using “calibration questions”
(the rubric) also created by the instructor. The CPR software assigns a reviewer competency
score based on a comparison of the student review to the instructor review of each essay.
Students then review three classmates’ essays (randomly assigned and anonymous) and
their own essays, using the same calibration questions.
Instructor-reported experiences and a limited number of studies have suggested that CPR is
a tool that can help students master content, improve writing skills, and become more
competent reviewers (Furman & Robinson, 2003; Margerum et al., 2007; McCarty, Parkes,
Anderson, Mines, Skipper, & Greboksy, 2005; Pelaez, 2002; Russell, 2001). Gerdeman,
Russell, and Worden (2007) examined the development of 1330 students’ writing and
reviewing skills in an introductory biology course and found that students showed
improvement in writing and reviewing over three CPR assignments. The design of CPR was
motivated by a belief that writing and peer review can help students learn content and
critical thinking skills. Research by educational researchers and practitioners has
demonstrated that having students write (e.g., Barnett & Blumner, 1999; Herrington, 1997;
Klein, 1999; Lowman, 1996; Paul, 1995; Rivard, Stanley, & Straw, 2000; Sternberg, 1994;
Wright, Herteis, & Abernethy, 2001) and review each other’s work (e.g., Boud, 1990; Cutler
& Price, 1995; Dochy, Segers, & Sluijman, 1999; Orsmond, Merry, & Callaghan, 2004;
Pope, 2005; Reese-Durham, 2005; Sobral, 1997; Topping, 1998) are effective ways of
teaching and learning. It should be noted our aim with this study is not to investigate the
usefulness of CPR, but to find out student reactions as well as the steps taken to overcome
student resistance to an innovative technological tool.
Background Information
The focus of the study is a first-year general chemistry class which is a two-semester
sequence involving almost 3000 students each semester at Texas A&M University (TAMU).
Students attend three hours of lecture per week in a class of 250-300 students and
participate in a weekly three-hour lab in sections of 24 students, taught by graduate
teaching assistants.
In Spring 2002, CPR was introduced to faculty teaching in the First Year Chemistry Program
during an NSF-sponsored Multi-Initiative Dissemination workshop. The instructor had always
included writing in her large chemistry classes, but saw CPR as a way to increase the
amount of writing without additional graders. She convinced the program's director that CPR
was worth trying and the decision was made to implement CPR in all sections of general
chemistry beginning in Fall 2002. This study involves only the students in her sections. At
the end of each semester of implementation, she collected student feedback with Student
Assessment of Learning Gains (SALG, http://www.wcer.wisc.edu/salgains/instructor/), a
customizable web-based program designed to capture student perceptions of their learning
gains during a given course.
https://doi.org/10.20429/ijsotl.2008.020105
3
Overcoming Student Resistance to a Teaching Innovation
Methods
The specific research questions addressed in this study are:
(1) What do students think about CPR as a learning tool?
(2) What do student comments reveal about the reasons for their acceptance of or
resistance to CPR?
(3) Why and how did the instructor persist?
While quantitative data from the online survey, SALG, were used to investigate the first
research question, qualitative data (student comments) from the survey were used to
investigate the second research question. During the analysis of student comments, the
third research question, focusing on the instructor’s experience, arose. The instructor in our
study (and the first author of this paper) was initially motivated to use CPR because she
believed that writing could help her chemistry students learn essential course content and
that peer review could help develop their critical thinking skills. The resistance she
encountered, while understood in retrospect, was unexpected. Despite student angst and
even antagonism about CPR, the instructor persisted and we (the faculty developers)
wanted to know why. In addition, the nature of the student comments gradually changed
and we wanted to know if there were modifications the instructor made that could account
for this change. To address these questions, we asked the instructor to reflect and write
about her implementation of CPR. Her reflections and answers to the third research question
are presented in the findings section.
Participants in the study were the 1515 students in the instructor’s sections of first-year
general chemistry during seven semesters (Fall 2002 - Spring 2006, excluding Fall 2003).
Students were asked to complete the SALG survey several days before their final exam;
completion was worth 5 points on their final. Students logged into the web-based program
SALG with their names so credit could be given, but SALG dissociated their names from
their responses, maintaining anonymity while motivating students to complete the
assessment. The response rate ranged from 94-98% over the seven semesters.
Research Question 1: Quantitative Analysis
In order to investigate what students thought of CPR as a learning tool, we conducted a
quantitative analysis of the five SALG survey items pertaining specifically to CPR:
1. Do you think that future classes should do CPR? Please explain.
2. I enjoyed doing the CPR assignments.
3. The CPR assignments helped me learn some chemistry.
4. The CPR assignments helped me improve my writing skills.
5. The CPR assignments helped me learn to critique my own writing and that of
others.
While students answered item 1 with a yes/no response with further explanation, they rated
items 2-5 on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree).
Quantitative analysis included: (1) the tabulation of responses to items 1-5, (2)
correlational analysis among responses to items 2-5, (3) analysis of the relationship of
https://doi.org/10.20429/ijsotl.2008.020105
4
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
items 2-5 to item 1, and (4) chi-square analysis on pairs of items 2-5 for each semester.
We also wanted to investigate whether students’ suggestions that future classes use CPR
(positive responses to item 1) necessarily reflected a positive experience with CPR, which
would be summarized in their responses to items 2-5. Thus, we averaged the responses to
items 2-5 for each student and interpreted an average of less than 3 to be an overall
negative experience and greater than 3 to be a positive experience. Then, we calculated the
percentage of students in each semester that fell into the following four groups:
Group 1: Students who had a negative experience, yet wanted future classes to do
CPR.
Group 2: Students who had a negative experience and did not want future classes
to do CPR.
Group 3: Students who had a positive experience and wanted future classes to do
CPR.
Group 4: Students who had a positive experience, yet did not want future classes to
do CPR.
Finally, for each group we counted the number of positive and negative statements obtained
in the qualitative analysis to observe if the quantitative determination of their positive or
negative CPR experience was linked to either their qualitative responses or their wish to
have future classes do CPR.
Research Question 2: Qualitative Analysis
Qualitative responses ranged from phrases to paragraphs written as responses to item 1,
“Do you think future classes should use CPR?” From the 1515 students, 1264 provided
explanations to this question; 37 explanations were ambiguous, such as “I don’t care,” and
were not included in the analysis. Using qualitative methods of analysis, we looked for
patterns and themes in these explanations that investigated the reasons for student
acceptance of or resistance to CPR. We read through all responses from one semester and
came to consensus about what categories we would use and how each statement would be
coded. The categories that emerged reflected our desire to understand why students liked
or did not like CPR and in what ways they perceived it to affect their learning. The
participation of the instructor in the coding process was critical because she was able to
provide contextual information that helped us to understand the student comments. After
coding each semester’s data we refined our categories, adding new ones where needed and
combining others. After coding the entire data set, we tabulated the results.
Research Question 3: Semester By Semester Implementation
Below is the instructor’s description (in her own voice) of how the implementation of CPR in
her first-year chemistry class evolved. While this section contains minimal explanation, a
detailed reflection of how and why she modified implementation strategies are presented in
the findings.
Fall 2002
Initially, I recognized that I could not ask students to do CPR in addition to all of the work I
had traditionally expected. To allow time for CPR, I decreased the number of labs per
semester from ten to seven. My students completed four CPR assignments with only the
best three scores counting toward their course grade so no makeup assignments had to be
given. Each CPR assignment was equivalent to a lab report, making CPR worth 5% of the
https://doi.org/10.20429/ijsotl.2008.020105
5
Overcoming Student Resistance to a Teaching Innovation
total grade. Since CPR was originally a chemical education tool, there were many
assignments already available for use. Thus I took all CPR assignments from the CPR library
rather than designing assignments specifically for my students. I gave the students a short
two-page handout explaining CPR.
Spring 2003
I implemented CPR in the same way as Fall 2002 in terms of the number and weight of
assignments. After glancing at student feedback, I wrote a detailed four-page handout that
included sections on how to do well on CPR and how to interpret their CPR scores. I
reframed CPR in my discussions with the students as an alternative way of assessing their
chemistry capabilities; as something that could help, for example, those students who do
not do well on multiple-choice tests. Finally, I emphasized my availability to help all
students with technical elements of the CPR software.
Fall 2003
Family Educational Rights and Privacy Act (FERPA) issues led my department to decide that
all student data (identification numbers and grades) needed to be kept secure on a server
inside the Texas A&M firewall. Because UCLA and Texas A&M had a joint NSF grant involving
CPR, the university was able to put CPR on its own server and I became the university's CPR
administrator because of my experience with the program. In Fall 2003, my class did only
one assignment to test the new system and I did not ask them to provide feedback; thus,
this semester is not included in our study.
Spring 2004
Students completed three CPR assignments, with only the best two assignments counting
toward their grade, since I could only replace 2 labs that semester. The CPR assignments
were worth 3% of the total grade. Two important changes in implementation occurred
during this semester. First, instead of using only assignments available in the CPR library, I
wrote two of the assignments, "Plagiarism in TAMU Laboratories" and "Measurement &
Significant Figures." Second, I stated clearly that, when asked, I would be happy to look
carefully at peer-ratings and adjust scores if warranted.
Fall 2004
Changes made for this semester were in support of my attempt to meet the university
guidelines of a "W" course and included both adjustments to grading policies and changes in
presentation of CPR to students. I told my students that my class was writing-intensive on
the first day and included this fact in my syllabus. My students returned to doing three
fewer laboratories than students in other sections. To emphasize this, in the syllabus'
laboratory calendar, I marked the days without laboratories as “CPR lab holiday.” CPR now
counted for 12% of the total grade and was equivalent to an exam. I invited students to let
me review their essays before submitting them to CPR. To emphasize the importance of
quality essays, I changed the CPR assignment grading weights to place more emphasis on
the text entry grade. I also began using Turnitin.com (www.turnitin.com) as an instructional
tool to help students check their work for plagiarism; each student was required to submit
the essay to Turnitin.com as a plagiarism check, and then resubmit to CPR. In this way, I
wasn't the "Plagiarism Police," but I was allowing students to check their own papers first. I
also took about 5 minutes per week from lecture to discuss common grammar mistakes and
included an extra question on grammar on their exams. I did meet all the requirements for
the "W" course, but one—I didn't teach chemistry majors—and so in future semesters I
discontinued the emphasis on teaching grammar in class, although grammar tips continued
to be included in the syllabus.
https://doi.org/10.20429/ijsotl.2008.020105
6
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Spring 2005
I continued the above activities and made increased efforts to communicate explicitly the
motivation and policies for using CPR. I added a copy of my teaching philosophy in my
syllabus. I spent class time to emphasize that CPR was one of several alternative ways for
students to learn and demonstrate their learning, so that students could struggle as testtakers and still do well in the course. I stated clearly that CPR was an assessment—a
demonstration of learning—in its own right, not simply a way to study for an exam.
Fall 2005
I continued to share my philosophy on teaching with my students. I took additional
classroom time to show students how to log in to both CPR and Turnitin.com. I used
Bloom’s taxonomy (Bloom et al., 1979) to help students recognize how important critiquing
skills were to their future. Throughout the semester, I emphasized that most students were
novice reviewers and that CPR was a tool to scaffold their reviewing skills. I stressed my
willingness to review, and override where warranted, their peers’ rating of their essays.
Spring 2006
I continued implementation of CPR as before, with all of the modifications I had previously
made. In addition, while my students were working on their first assignment, I added an inclass discussion on strategies for success on the calibration portion of the assignment.
Findings
Research Question 1: What do students think about CPR as a learning tool?
The first part of item 1 on the SALG asked students to indicate whether or not future classes
should use CPR. Table 1 presents percentages of students who responded yes or no to this
question during each of the seven semesters. The percent of students who believed that
future classes should use CPR rose from 43% in Fall 2002 to 71% in Spring 2006. Items 25 on the SALG asked students to indicate their agreement with statements about their
enjoyment of CPR and about the value of CPR to their learning. Table 2 shows percent
response to these items in each of three categories: strongly agree/agree, neutral,
disagree/strongly disagree. Graph 1 shows the percentage of those students who agreed or
strongly agreed with the items 1-5 for each semester. From the beginning, more students
understood the value of CPR for improving their ability to critically review (item 5) and by
Spring 2006, even though only 26% of the students enjoyed CPR as an exercise (item 2),
70% of them recognized that CPR helped them hone their reviewing skills.
SALG items 2 through 5 allowed us to break the question “What do students think about
CPR?” into several smaller questions. We investigated the relationships among these
questions for all semesters. When we examined the data, we found moderate but significant
positive correlations each semester among responses to items 2 through 5. Spearman
correlation coefficient rho values ranged from 0.48 to 0.70, significant at p<0.001 (2
tailed).
In order to investigate the Likert items further, chi square 2x2 contingency tables were
produced for each semester by dividing the students into two groups: those that agreed and
strongly agreed with an item and those that were neutral, disagreed or strongly disagreed
with the item. In the chi-square analysis, we found that the values of X2(1) between all
pairs ranged from 8 to 53 (Table 3), showing that the data were interdependent at
p<0.005. This interdependence is reflected by the 2x2 contingency table results for each
semester (Table 4). Using Spring 2006 data as an example, we saw that of the 60 students
https://doi.org/10.20429/ijsotl.2008.020105
7
Overcoming Student Resistance to a Teaching Innovation
(26%) who reported enjoying CPR, 90.0% indicated that CPR helped them learn chemistry,
78% that it improved their writing and 95% that it improved their critiquing skills; and of
the 175 students (74%) who did not enjoy CPR, only 47% thought it helped them learn
chemistry, 33% that it improved their writing, and 62% that it improved critiquing skills.
Using odds ratios (Graph 2), we found that all students in the study who enjoyed CPR were
on average 10 ± 5 (std. dev.) times more likely to think CPR helped them learn more
chemistry than those who didn't enjoy CPR, 11 ± 6 times more likely to think CPR improved
writing skills, and 9 ± 5 times more likely to think CPR improved critiquing skills than those
students who did not enjoy CPR.
We compared the quantitative measurement of student experience with CPR with whether or
not students recommended CPR to future classes (Graph 3). The percentage of students with
a negative experience (average of <3 on items 2-5) who wanted future students to use CPR
(Group 1) stayed relatively small at 11.2 ± 1.5%. Also, the percentage of students who saw
some benefit to CPR (average >3 on items 2-5) but did not believe future classes
should use it (Group 4) stayed constant at 3.3 ± 0.6%. However with time and the
instructor's gained experience, the percentage of students who had a negative experience
and hoped that future classes would not have to use CPR (Group 2) decreased, and the
percentage of students who had a positive experience and did want future classes to use
CPR (Group 3) increased from 25% to 54%. Findings from our qualitative analysis helped to
interpret these results.
Research Question 2: What do student comments reveal about the reasons for
their acceptance of or resistance to CPR?
Student explanations to item 1 (in response to why students recommend or not that CPR be
used in future semesters) provided insight into their acceptance of or resistance to CPR.
During the analysis of 1227 explanations, we focused on the statements within each
explanation. An explanation could contain one or more statements. For example, “I don’t
think my peers should grade me, but I thought the overall CPR process was useful in
improving writing skills; it also helped me learn chemistry,” is one explanation with three
statements, one negative and two positive.
Throughout the analysis, we let categories emerge from the statement or statements in the
explanations with a focus on what students were actually saying. We collected positive
categories in three groups: (A) CPR helped learning in the ways that the instructor intended,
(B) CPR was beneficial to student development in ways that the instructor did not explicitly
intend and (C) CPR was viewed positively for reasons not necessarily tied to learning. Four
subcategories of A, enhanced learning of critical content, enhanced critical thinking skills,
enhanced writing skills, and helped link chemistry to life were benefits that the instructor
had intended for her class. Three subcategories of B, prepared students for future and
professional life and developed time management and communication skills were benefits of
CPR that the students brought to our attention and the instructor had not intended. Finally,
two subcategories of C, better than labs and an alternative means to show learning were
grouped as benefits that were unrelated to learning. Table 5 presents the percentages of
positive categories for each semester.
Negative statements fell into four categories: (A) CPR did not help learning, (B) Complaints
about grading and peer review, (C) Writing does not belong in a chemistry class, and (D)
Other reasons for not liking CPR. Table 6 presents the percentages of negative categories
for each semester.
The following student response illustrates our analysis: “I don’t think my peers should grade
me, but I thought the overall CPR process was useful in improving writing skills. It also
https://doi.org/10.20429/ijsotl.2008.020105
8
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
helped me learn chemistry.” This answer contains three statements falling under three
categories: a negative category (complaints about grading and peer review) and two
positive categories (enhanced writing skills and enhanced learning of critical content).
Results indicated that the despite decreasing from the first semester to the second, the
percentage of positive statements steadily increased from Spring 2003 until the final
semester (Table 7). In fact, the majority of statements fell under positive categories in the
last three semesters (51% in Spring 2005, 55% in Fall 2005, and 56% in Spring 2006). In
the next section, we explain and illustrate the three groups of positive categories and four
negative categories. The percentages for each semester are presented in Tables 5 and 6.
Category Analysis of Positive Statements
Category A (CPR helped learning in the ways that the instructor intended)
Some students whose statements fell under this group indicated that CPR helped them learn
critical course content while some commented that CPR improved writing skills. In fact,
these two
subcategories had the highest percentage of positive statements in each semester, and
although the percentage fluctuated, it remained fairly high. Some students observed that in
order to write about a topic, you must understand and learn it: “With CPR there's no way
around learning the information. To write a good paragraph, you're going to have to know
what you're talking about;” “CPR really helped me understand the topics. It reinforced the
material by forcing me to teach myself and explain it to others through writing. It was very
helpful.”
One student wrote:
Calibrated Peer Review forces the student to look into the topic way more closer [sic]
than what he or she would do out of a textbook. I know the CPR has tremendously
helped me understand each topic better although I didn't exactly enjoy it so much.
Some students commented that CPR helped them develop not only general writing skills, but
specifically scientific writing skills. In fact, some pointed out that CPR was the first time that
they had to practice discipline-specific writing at all: “I'm an engineering student and this
was the only time I was ever really exposed to writing this semester so it kind of practiced
my writing skills;” “Even though I didn't want to use it, it was my first exposure to technical
writing.”
Students also noted that reviewing others’ work helped their understanding and enhanced
their critical thinking skills: “Critiquing the other students helps you see what you did wrong
and helps you understand better;” “It does help you learn how to look critically at other
people's and even your own writing.”
Although a lesser percentage, some students actually commented that writing helped link
chemistry to life: “CPR assignments help students to better connect chemistry to aspects of
real life;” “CPR helps with writing skills and also helps you relate chemistry topics with
things in the outside world!”
Category B (CPR was beneficial to student development in ways that the instructor did not
explicitly intend)
While statements in the previous group described the learning benefits for which CPR was
designed, a small percentage of students commented on learning that had not been
explicitly intended. For example, some students noted that using CPR helped them to
https://doi.org/10.20429/ijsotl.2008.020105
9
Overcoming Student Resistance to a Teaching Innovation
prepare for future and professional life: “CPR should be required for all science classes
because it helps the student frame their work in the proper perspective of that field.” Others
pointed out that CPR helped develop specific skills such as time management and
communication skills: “If anything, it teaches kids to be responsible with their assignments
and time deadlines and to incorporate other subjects (writing) with Chemistry.”
Category C (CPR was viewed positively for reasons not necessarily tied to learning) Some
of the positive statements were unrelated to student learning and development. Some
students simply noted that doing a CPR assignment was preferable to doing a lab. While
the percentage of students who made this statement was high in Fall 2002 (25%) it
decreased to 0-5% in the other semesters. Some students noted that CPR helped their
grade and appreciated that it was an alternative means of demonstrating what they were
learning: “CPR allows another opportunity, other than exams, to test your knowledge on the
subject being tested on. Additionally, it provides another way to boost your grade.” The
percentage of students who noted that CPR was helpful as an alternative method of
assessment increased from 2002 to 2006.
Category Analysis of Negative Statements
Category A (CPR did not help learning)
Students who noted that CPR did not help their learning indicated that it did not help them
develop their writing skills or learn chemistry. The percentage of this category varied but did
slightly decrease from 2002 to 2006. Although in most cases, students simply made
statements such as “it did not improve my writing skills” without further explanation,
statements under categories B and C described below helped explain student resistance to
CPR.
Category B (Complaints about grading and peer review)
Statements in this category reflected student expectations that their grades should be the
domain of the instructor. While some complaints were about CPR’s grading system in
general, such as “The grading system could be made more fairly” and “The way they are
graded is not fair,” some students reported being uncomfortable with the idea of someone
at their same knowledge level assigning a grade: “Why would people comment on papers
and grade them when they are in the process of learning the material themselves?”
Students indicated a belief that the grading was too subjective and that judgment of writing
was only an opinion and that they should not be penalized if their opinion was different from
their peers. For example, one student wrote: “They ask you to grade the essays, but then
your opinion of how that person did would be wrong. I just don't see how your opinion could
be wrong.” Another student wrote:
The way that the grading system works is due very much to personal discretion
and open to ones own interpretation. There were times that I would find something
wrong and marked it that way yet had reasoning behind it and had to redo the
entire reviewing of that essay.
Category C (Writing does not belong in a chemistry class)
Statements in this category reflected student expectation that writing belongs in English,
not in science classes. Some indicated that writing and peer critique are not necessary for
chemistry: “I have never viewed chemistry as being a subject where you write things;” “We
could take English to learn how to write correctly;” “I didn't understand why writing a paper
and grading other students papers had anything to do with chemistry.” In Fall 2004, when
the instructor was attempting to make her class a "W" course, 21% of the negative
statements were in this category. Before then, when CPR assignments were a part of the
lab and worth on 3-5% of the grade, students made fewer statements that fell under this
https://doi.org/10.20429/ijsotl.2008.020105
10
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
category. After Fall 2004, the comments again decreased because of instructional
modifications.
Category D (Other reasons for not liking CPR)
The largest percentage of negative statements fell into this category consisting of
statements about CPR that had, on the surface, little to do with learning. Students wrote
that CPR was too time consuming, harmed their grade and was worse than labs, caused or
added to stress, and that meeting deadlines was a problem. In contrast to the first three
categories, statements in category D could be heard about almost any course requirement.
They reflect a need to help students accept that learning requires time, is often stressful,
and that deadlines are a part of life. What we cannot say for certain based on this analysis
is whether the time-consuming nature of CPR prevents students from seeing its benefit to
learning, or if not seeing the benefit leads to the conclusion that it takes too much time. We
can say, from a separate survey, that 94±2% of the students over seven semesters said
the amount of time allowed for CPR was adequate.
Further Analysis of Quantitative Data Using Supporting Qualitative Data Before
proceeding to the third research question, let us revisit our earlier quantitative analysis of
student experience with CPR as it relates to whether or not the students recommended CPR
to future classes (Graph 3). After conducting the qualitative analysis, we counted the
number of positive and negative statements for each group (Table 8 and Graph
4) and the results were surprising. We had interpreted students in Group 1 (whose
responses to items 2-5 indicated a negative experience but who said that future students
should use CPR) to be communicating a desire to make future students “suffer” as they had.
The qualitative analysis did not support this interpretation. With the exception of the first
semester, students in Group 1 made three times more positive statements than negative
statements. Thus, it appears that this group is not saying “if we had to do it so should
they.” Rather, they are indicating that while it had not enhanced their personal learning
experience, they could see that it could be a beneficial learning tool. Similarly, students in
Group 4 (whose responses to items 2-5 indicated a positive personal experience but who did
not indicate that future students should use CPR) made three times more negative
statements than positive ones. Students in Group 2 who had an overall negative experience
with CPR and who did not want future classes to do CPR gave six times more negative
statements than positive ones, and students in Group 3 with an overall positive experience
with CPR and who did want future classes to do CPR gave seven times more positive
statements than negative ones.
Research Question 3: Why and how did the instructor persist?
This section presents the instructor’s reflection (in her own voice) of her use of CPR
throughout the semesters. This detailed report reveals how and why she persisted in using
CPR in her classes.
General Reflection
If I had been attuned to the literature on introducing innovative learning tools to the
classroom, I would have expected the resistance I experienced. However, the level of
student unhappiness was totally unexpected. Using the SALG feedback and simple
conversations with students, I slowly began to incorporate changes into the class to lessen
student angst by intuitively using the process of reflection-in-action (Schön, 1983). Table 9
shows the reflective path I took combined with actual qualitative data gleaned from our
study, even though I did not explicitly study the data at the time.
https://doi.org/10.20429/ijsotl.2008.020105
11
Overcoming Student Resistance to a Teaching Innovation
Fall 2002
I knew CPR was an excellent program, but I was a novice user. I assumed it was selfexplanatory and that my students were so computer-savvy that they would have no
problems. I also presumed that the grading part of the program would not need any
intervention. However, as I listened to the students and read SALG comments, I learned that
students were confused about how CPR worked and how the assignments were graded.
Spring 2003
I realized I had naively assumed that the instructions available on the CPR website were
sufficient to explain the process, but student complaints indicated otherwise. So, before the
semester began, I prepared an extensive four-page handout that thoroughly discussed how
the system worked, how to do well, and how to read the grading page. After listening to the
students, I also began to emphasize that I would help anyone with problems with the
software. I started to consider CPR as an alternative for showing chemistry proficiency and I
discussed this in class. CPR was only worth 5% of their grade at that point, but at the end
of the semester, there were some student comments stating that CPR enhanced learning. I
noticed that although the complaints about instruction diminished, grading complaints were
still high.
Spring 2004
At this point, I worked very hard to show that the grading was fair. I invited students to send
me an email requesting a grade check if they thought they were graded unfairly. I began to
proactively look at student grades and change them when deserved. I wanted to help those
students who thought, and reported, that they were not learning from CPR and I thought
that if the assignments were written by me, students would feel more comfortable. Thus, I
began to write my own assignments. The results were encouraging, as I noticed that there
were no grading complaints, outside of those related to peer reviewing. Now I wanted to
address the objections that writing was not fit for a chemistry class, which was also a
common theme in student criticisms.
Fall 2004
I began the semester by stating that my class was a “writing intensive” class and increased
the CPR component to 12% of the grade, equal to that of an exam. I noticed that there was
an increase in student comments stating that CPR helped their writing skills, probably due
to the extra emphasis I placed on teaching grammar that semester. Students started
commenting more that CPR helped bad test takers. However, students still said that writing
was not appropriate for a chemistry class.
Spring 2005
In my syllabus, I continued to emphasize my policy of incorporating other ways that would
enable my students to be successful. I added my teaching philosophy to the syllabus to
share my motivation for my teaching practices and establish trust between my students and
me. My teaching philosophy stated that 47% of the grade was from work other than exams
and that CPR was not merely preparation for exams, but was an actual grade. By the
semester’s end, students knew CPR could help their grade if they were bad test takers. In
addition, statements indicating that CPR enhanced learning of critical content increased.
However, students also realized that the CPR grade was important and could actually harm
their average, which led to more negative comments regarding grading issues, including
peer review problems. Meanwhile, comments suggested that students still did not think that
writing had a place in a chemistry class.
Fall 2005
https://doi.org/10.20429/ijsotl.2008.020105
12
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
In order to assist students further in technological issues, I gave a short presentation on
how to log into CPR in class. In addition, to address the place of writing in chemistry, I used
the principles of Bloom’s Taxonomy to illustrate the importance of writing and critiquing in
chemistry and for future careers. Throughout the semester, I emphasized that students
were novice reviewers and that I would review student papers before submission if asked. I
also assured them I would regrade their papers if necessary. For the first time, I noticed a
decrease in comments suggesting that writing was not fit for a chemistry class and
complaining about general grading issues.
Spring 2006
I was still concerned about student resistance to the grading process, so not only did I
demonstrate how to write an essay for CPR, I also showed the students how to critique
others using a grading rubric. Finally, students no longer complained about grading and
comments suggesting that writing was not fit for a chemistry class lessened. Meanwhile,
there was an increase in comments stating that CPR enhanced learning.
Discussion
As the findings indicate, the answers to the research questions (What do students think
about CPR as a learning tool? What do student comments reveal about the reasons for their
acceptance of or resistance to CPR? How and why did the instructor persist, particularly in
the face of initial student resistance?) are closely connected. The quantitative results
indicated that initially the majority of students did not like CPR and did not believe that it
helped their learning. The qualitative analysis revealed that this student resistance to CPR
was accompanied by a strong sense that writing and reviewing have no place in a chemistry
class. In addition, student resistance was exacerbated by student distrust of the ability of
their peers to review fairly and accurately.
However, despite their initial resistance, student perceptions changed over time and became
more positive. Over the seven semesters, the percentage of students in each class who
enjoyed CPR rose from 11% to 26%; those who recognized that CPR helped in learning
chemistry rose from 21% to 58%; those who perceived a gain in writing skills rose from
28% to 45%; and those who recognized that they gained critiquing skills rose from 43% to
70% (Table 2 and Graph 1). Simultaneously, the percentage of positive statements steadily
increased from Spring 2003 until the final semester (Table 7). In fact, the majority of
statements fell under positive categories in the last three semesters (51% in Spring 2005,
55% in Fall 2005, and 56% in Spring 2006).
It is also encouraging that despite the resistance to CPR, the percentage of positive
statements indicating that CPR enhanced writing skills did not fall under 28% in any
semester and even went up to 51% in Fall 2004, when the instructor attempted to have her
course classified as a university "W" course (Table 5). Meanwhile, the percentage of positive
statements indicating that CPR enhanced learning of critical content did not fall under 20%.
Since the increase in overall positive comments happened as the instructor modified her
implementation strategies, we relate the former to the latter. Akerlind and Trevitt (1999)
suggest that an innovative technological tool may impact students directly through its use
and indirectly through its effect on other aspects of the course, which is what happened in
the instructor’s course. Through the use of CPR, the instructor started changing not only the
structure and focus of the course itself, but also how she presented herself as an educator,
such as sharing her own teaching philosophy with her students. While the instructor had not
been doing thorough analysis of SALG data until 2006 when we began this study, she was
https://doi.org/10.20429/ijsotl.2008.020105
13
Overcoming Student Resistance to a Teaching Innovation
scanning the results and reading student comments. Her own belief in the value of CPR was
strong enough to be encouraged by positive comments and to resist giving up. The
instructor’s practice of collecting student perception data through SALG kept her informed
about student resistance and anxiety so that she could make targeted improvements. This
process, called reflection-in-practice (Schön, 1983) was used intuitively by the instructor.
Although initially surprised by the students' negativity toward CPR, she changed the class
structure and her teaching style to alter student attitude. She used negative comments to
guide her future use of CPR. She recognized that, until the introduction of CPR, it was rare
for chemistry students to be asked to write essays, and rarer still for them to be expected to
review each others’ writing. Thus, through the lens of the Theory of Perceived Attributes
(Rogers, 2005), CPR did not have “compatibility”: The nature of CPR assignments ran
counter to what students expected. The comments made her realize that CPR had neither
relative advantage nor “observability” for the students—students could not observe and did
not realize its benefits. Thus, she needed to help students see its value and the abilities it
fosters. For example, instead of simply assuming that students would recognize the
importance of learning to review, she tried to create “change through persuasion” (Akerlind
& Trevitt, 1999, p. 101) by discussing the importance of learning to review with them during
class and connecting it to the kinds of careers they would want to have in the future. She
also started giving advice about how to succeed in CPR assignments, reassuring the
students that she would help them with technical issues if necessary, and providing more
guidelines on the software and examples of how to use it during class. After the first two
semesters, she became more involved with the CPR software by creating her own
assignments and provided support that would relieve student anxiety about peer grading.
She became much more intentional about communicating to the students what they would
gain from completing CPR assignments and tried to instigate “attitudinal and conceptual
change” (Akerlind & Trevitt, 1999, p. 101) by presenting her deep-seated student-centered
teaching philosophy in her syllabus and abiding by that philosophy to obtain student trust.
This study demonstrated the power of a mixed methods approach when we first correlated a
quantitative measurement of student experience (the numerical average of Likert scale
items 2-5) with their recommending CPR to future classes. From the quantitative data
alone, we were prepared to say that students who did not have an overall positive CPR
experience and wanted future classes to do CPR, were actually saying "if we had to, then
they have to," in a negative way. This was not the case, as 80% of their comments were
positive (Graph 4). Supporting qualitative data prevented us from misinterpreting the data.
While it is not necessary for students to “like” a particular learning tool in order to benefit
from it, the correlational analysis of the data demonstrated that students who enjoyed CPR
reported that they received 4-22 times more benefit (with regard to their learning, writing
skills and critiquing skills) from it than those who did not. Although only a small percentage
of the students admitted that they enjoyed CPR, we can say that when students understand
the value of CPR, they are more likely to see that it improves learning, writing, and
critiquing skills. But why was student resistance so persistent despite the changes that the
instructor made and the increase in positive student reactions? Akerlind and Trevitt (1999)
suggest when an innovative tool is introduced, students must undergo a paradigm shift in
their perception of learning and readjust their notions about the roles of student and
instructor, which all lead to negative reactions. These perceptions are not easy to change,
since they are culturally rooted beliefs that begin in elementary school (Cuban, 1993).
This study has implications for the introduction of any teaching innovation. Akerlind and
Trevitt (1999) suggest that instructors must expect student resistance to such innovations
and be ready to work through the student resistance to innovations and not be discouraged.
In addition to this, it is important that students be coached through the adaptation and have
https://doi.org/10.20429/ijsotl.2008.020105
14
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
enough guidance regarding the software itself, since perceived complexity of a software
program is negatively correlated with perceived enjoyment and perceived usefulness of the
tool (Cheung & Huang, 2005). Our instructor believed in the benefits of CPR and
immediately began to provide more help to the students in the next semester, but some aid
was more beneficial than others in decreasing negative responses. She found that direct
face-to-face help, for example, in-class demonstrations and office hours for aiding with
software issues, was much more beneficial to the students than indirect help, for example,
extensive handouts.
When the instructor in our study explicitly conveyed to the students the value of writing and
peer critiquing for learning chemistry and for their future careers and made the CPR
assignments worth a significant part of their grade, students reported a much more positive
experience with this particular innovation. This study demonstrated that students are willing
to take a more active and responsible role—even when the innovation runs counter to their
expectations—when they perceive the value of such engagement and are supported in their
efforts.
References
Akerlind, G. S. & Trevitt, A. C. (1999). Enhancing self-directed learning through educational
technology: When students resist the change. Innovations in Education and Training
International, 6(2), 96-105.
Barnett, R. W., & Blumner, J. S. (Eds). (1999). Writing centers and writing across the
curriculum programs. Westport, CT: Greenwood Press.
Bloom, B.S. (Ed.). (1979). Taxonomy of educational objectives: Book 1, Cognitive domain.
London: Longman.
Boud, D. (1981). Developing Student Autonomy in Learning. London: Kogan Page.
Boud, D. (1990). Assessment and the promotion of academic values. Studies in Higher
Education, 15(1), 101-111.
Cheung, W., & Huang, W. (2005). Proposing a framework to assess Internet usage in
university education: An empirical investigation from a student’s perspective. British Journal
of Educational Technology, 36(2), 237–253.
Cuban, L. (1993). Computers meet classroom: Classroom wins. Teachers College Record,
95(2), 185-210.
Cutler, H., & Price, J. (1995). The development of skills through peer assessment. In A.
Edwards & P. Knight (Eds.), Assessing Competence in Higher Education (pp. 150-159).
London: Kogan Page.
Dochy, F., Segers, M., & Sluijman, S. (1999). The use of self-, peer and co-assessment
in higher education: A review. Studies in Higher Education, 24, 331-350.
Falchikov, N. (1995). Peer feedback marking: Developing peer assessment. Innovations
in Education and Training International, 32(2), 175-187.
Furman, B., & Robinson, W. (2003). Improving engineering report writing with Calibrated
https://doi.org/10.20429/ijsotl.2008.020105
15
Overcoming Student Resistance to a Teaching Innovation
Peer Review. Paper presented at the 33rd ASEE/IEEE Frontiers in Education Conference,
November 5-8, 2003, Boulder, CO, pp. F3E-14-F3E-15.
Gerdeman, R. D., Russell, A. R., & Worden, K. J. (2007). Web-based student writing and
reviewing in a large biology lecture course. Journal of College Science Teaching (March/
April 2007), 46-52.
Herrington, A. J. (1997). Developing and responding to major writing projects.
In Sorcinelli, M. D., & Elbow, P. New Directions for Teaching and Learning: Writing to learn:
strategies for assigning and responding to writing across the disciplines (69). San Francisco:
Jossey-Bass.
Klein, P. D. (1999). Reopening inquiry into cognitive processes in writing-to-learn.
Educational Psychology Review, 11(3), 203-270.
Kovac, J. & Sherwood, D. W. (1999). Writing in chemistry: An effective learning tool.
Journal of Chemical Education, 76(10), 1399-1403.
Lowman, J. (1996). Assignments that promote learning. In R. J. Menges, M. Weimer,
& Associates (Eds.), Teaching on solid ground: Using scholarship to improve
practice. San Francisco: Jossey-Bass.
Margerum, L. D., Gulsrud, M., Manlapez, R., Rebong, R., & Love, A. (2007). Application of
calibrated peer review (CPR) writing assignments to enhance experiments with an
environmental chemistry focus. Journal of Chemical Education, 84(2), 292-295.
McCarty, T., Parkes, M. V., Anderson, T. T., Mines, J., Skipper, B. J., & Greboksy, J. (2005).
Improved patient notes from medical students during web-based teaching using facultycalibrated peer review and self-assessment. Acad Med, 80, 67-70.
Orsmond, P., Merry, S., & Callaghan, A. (2004). Implementation of a formative
assessment model incorporating peer and self-assessment. Innovations in
Education and Teaching International, 41(3), 273-290.
Pelaez, N. J. (2002). Problem-based writing with peer review improves academic
performance in physiology. Advanced Physiology Education, 26, 174-184.
Paul, R. (1995). Critical thinking: How to prepare students for a rapidly changing
world. Santa Rosa, CA: The Foundation for Critical Thinking.
Pope, N. K. (2005). The impact of stress in self- and peer assessment. Assessment &
Evaluation in Higher Education, 30(1), 51-63.
Reese-Durham, N. (2005). Peer evaluation as an active learning technique. Journal of
Instructional Psychology, 32(4), 338-345.
Rivard, L. P., Stanley, B., & Straw, S. B. (2000). The effect of talk and writing on
learning science: An exploratory study. Science Education, 84(5), 566-593.
Rogers, E. M. (2005). Diffusion of innovations (5th ed.). New York: Free Press.
Russell, A. (2001). The evaluation of CPR. Prepared for HP e-Education; Business
Development. Los Angeles: UCLA.
https://doi.org/10.20429/ijsotl.2008.020105
16
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Schön, D. A. (1983). The reflective practitioner – How professionals think in action. New
York: Basic Books, Inc.
Smith, H., Cooper, A., & Lancaster, L. (2002). Improving the quality of undergraduate
peer assessment: A case for student and staff development. Innovations in
Education and Teaching International, 39(1), 71–81.
Sobral, D. T. (1997). Improving learning skills: A self-help group approach. Higher
Education, 33, 39-50.
Sternberg, R. J. (1994). Diversifying instruction and assessment. The Educational Forum,
59(1), 47-52.
Surry, D. W. & Farquhar, J. D. (1997). Diffusion Theory and Instructional Technology.
Journal of Instructional Science and Technology, 2(1). Retrieved April 12, 2007 from
http://www.usq.edu.au/electpub/e-jist/docs/old/vol2no1/article2.htm
Topping, K. J. (1998). Peer assessment between students in colleges and universities.
Review of Educational Research, 68(3), 249-276.
Van Patten, J. J. (2000, July 24). New delivery systems for the 21st century. Paper
presented at the 2000 Worlds Future Society Future Focus, Houston, TX. ERIC Dcoument
Reproduction Service No. 455 667.
Vuorela, M. & Nummenmaa, L. (2004).How undergraduate students meet a new learning
environment. Computers in Human Behavior, 20(6), 763-777.
Williams, E. (1992). Student attitudes towards approaches to learning and assessment.
Assessment and Evaluation in Higher Education, 17, 45–58.
Wright, W. A., Herteis, E. M., & Abernethy, B. (2001). Learning through writing: a
compendium of assignments and techniques (revised). Halifax, Canada: Dalhousie
University.
Table 1
Item1: “Do you think that future classes should do CPR™?”
Semester
No. of Responses
Yes
No
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
218
188
219
201
243
209
237
43%
36%
44%
57%
59%
68%
71%
57%
64%
56%
43%
41%
32%
29%
https://doi.org/10.20429/ijsotl.2008.020105
17
Overcoming Student Resistance to a Teaching Innovation
Table 2 - Likert Scale Data for Items 2-5
Item2: “I enjoyed doing the CPR assignments.”
Semester
Strongly
Agree/Agree
Neutral
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
11%
6%
10%
18%
19%
20%
26%
17%
16%
23%
22%
24%
25%
27%
Disagree/Strongly
Disagree
72%
79%
66%
59%
58%
55%
48%
Item3: “The CPR assignments helped me learn some chemistry.”
Semester
Strongly
Agree/Agree
Neutral
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
21%
20%
23%
34%
46%
45%
58%
26%
18%
20%
24%
21%
24%
21%
Disagree/Strongly
Disagree
53%
62%
57%
41%
33%
31%
21%
Item4: “The CPR assignments helped me improve my writing skills.”
Semester
Strongly
Agree/Agree
Neutral
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
28%
32%
28%
39%
41%
40%
45%
31%
29%
27%
23%
24%
27%
31%
Disagree/Strongly
Disagree
42%
40%
45%
38%
35%
32%
25%
Item5: “The CPR assignments helped me learn to critique my own writing and that of others.”
Semester
Strongly
Agree/Agree
Neutral
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
43%
39%
51%
60%
64%
64%
70%
24%
27%
24%
19%
11%
14%
21%
https://doi.org/10.20429/ijsotl.2008.020105
Disagree/Strongly
Disagree
33%
34%
25%
20%
26%
22%
9%
18
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Table 3
Chi-Square Data for Items 2-5 for Each Semester
Χ2(1) between enjoying CPR
and learning chemistry
2
Χ (1) between enjoying CPR
and improving writing
2
Χ (1) between enjoying CPR
and improving
critiquing
Χ2(1) between learning
chemistry and
improving writing
Χ2(1) between learning
chemistry and
improving critiquing
Χ2(1) between improving
writing and improving
critiquing
Fall
2002
Spring
2003
Spring
2004
Fall
2004
Spring
2005
Fall
2005
Spring
2006
8.0
(p<0.0
05)*
14.2
41.6
33.3
42.4
22.5
34.1
14.1
13.7
34.7
48.5
38.5
23.1
36.9
16.4
9.0
9.4
12.6
24.6
10.0
23.7
42.9
41.6
38.4
40.1
49.0
36.2
45.0
21.3
22.3
18.0
31.5
49.5
18.2
31.8
31.1
45.7
52.4
33.9
51.9
42.4
52.6
* All the rest were interdependent at p<0.001.
Table 4
Chi-Square 2x2 Contingency Table Data Comparing Students Who Enjoyed CPR to Students Who Did
NotEnjoyCPR
Fall
2002
Students who
enjoyed CPR
CPR helped them learn
chemistry
CPR improved their
writing
CPR improved critiquing
skills
Students who did not
enjoy CPR
CPR helped them learn
chemistry
CPR improved their
writing
CPR improved critiquing
skills
https://doi.org/10.20429/ijsotl.2008.020105
Spring
2003
Spring
2004
Fall
2004
Spring
2005
Fall
2005
11%
(N=23)
6%
(N=11)
10%
(N=22)
18%
(N=37)
19%
(N=45)
43%
64%
77%
76%
89%
78%
90%
61%
82%
82%
89%
82%
73%
78%
83%
82%
82%
86%
96%
85%
95%
89%
(N=193)
94%
(N=176)
90%
(N=194)
82%
(N=164)
81%
(N=198)
20%
(N=41)
Spring
2006
80%
(N=168)
26%
(N=60)
74%
(N=175)
18%
16%
16%
26%
35%
37%
47%
24%
27%
22%
27%
32%
32%
33%
38%
34%
47%
55%
56%
59%
62%
19
Overcoming Student Resistance to a Teaching Innovation
Table 5
Category and Subcategory Data for Positive Statements
Semester
Total
A1
A2
A3
B1
B2
B3
C1
C2
Fall 2002
Spring
2003
Spring
2004
Fall 2004
Spring
2005
Fall 2005
Spring
2006
Total
69
35
33%
31%
10%
11%
28%
49%
1%
0
3%
6%
0
3%
25%
0
0
0
82
20%
18%
45%
0
9%
1%
5%
2%
72
121
29%
44%
11%
9%
51%
28%
1%
1%
0
0
1%
7%
0
1%
7%
10%
96
116
36%
32%
10%
13%
30%
34%
4%
1%
5%
6%
0
0
1%
3%
14%
11%
591
A1= enhanced learning of critical content
A2 = enhanced critical thinking skills
A3 = enhanced writing skills
B1 = helped link chemistry to life
B2 = prepared students for future and professional life
B3 = developed time management and communication skills
C1 = better than labs
C2 = an alternative means to show learning
Table 6
Category and Subcategory Data for Negative Statements
Semester
Total
A
B1
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
Total
162
152
131
105
115
80
93
838
21%
26%
17%
17%
14%
21%
11%
14%
10%
8%
2%
15%
7%
5%
B2
C
D
6%
3%
5%
6%
5%
5%
10%
6%
10%
14%
21%
16%
9%
4%
53%
51%
56%
54%
50%
58%
70%
A = CPR did not help learning
B1 = complaints about grading
B2 = complaints about peer review
C = writing does not belong in a chemistry class
D = other reasons for not liking CPR
https://doi.org/10.20429/ijsotl.2008.020105
20
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Table 7
Percentages of Positive and Negative Statements
Semester
Total
Statements
Positive
Statements
Positive
Statements
(%)
Fall 2002
Spring 2003
Spring 2004
Fall 2004
Spring 2005
Fall 2005
Spring 2006
Total
231
187
213
177
236
176
209
1429
69
35
82
72
121
96
116
591
30%
19%
38%
41%
51%
55%
56%
Negative
Statements
Negative
Statements
(%)
162
152
131
105
115
80
93
838
70%
81%
62%
59%
49%
45%
44%
Table 8
Quantitative&QualitativeDataRegardingStudentExperiencewithCPR&ItsPromotionforFutureUse
Semester
No. of
Student
Responses
Group 1 (%)
(P/N)*
Group 2 (%)
(P/N)*
Group 3 (%)
(P/N)*
Group 4
(%)
(P/N)*
Fall 2002
218
11.0
(10P/11N)
48.2
(19P/85N)
25.2 (42P/13N)
3.2
(0P/7N)
Spring
2003
188
11.2 (12P/2N)
56.4
(10P/92N)
22.9 (25P/7N)
3.2
(1P/5N)
Spring
2004
219
10.0 (10P/3N)
47.0
(10P/81N)
26.5 (48P/7N)
3.7
(2P/6N)
Fall 2004
201
13.4 (20P/3N)
35.3 (8P/59N)
42.4 (66P/5N)
2.0
(1P/3N)
Spring
2005
243
9.5 (15P/5N)
35.0
(12P/70N)
42.4 (84P/8N)
3.3
(4P/6N)
Fall 2005
209
12.9 (20P/6N)
25.8 (7P/46N)
47.8 (78P/4N)
3.8
(4P/8N)
Spring
2006
237
9.7 (17P/4N)
22.4
(10P/42N)
54.9
(111P/19N)
3.8
(1P/6N)
Notes. * P is the number of positive statements; N is the number of negative statements.
The percentages do not add to 100% since there were students who averaged exactly 3.00 on items
1-4 who were omitted and not all students responded to every question.
Group
Group
Group
Group
1:
2:
3:
4:
Students
Students
Students
Students
https://doi.org/10.20429/ijsotl.2008.020105
with
with
with
with
negative CPR experience, yet want future classes to do CPR
negative CPR experience, and do not want future classes to do CPR
positive CPR experience, and want future classes to do CPR
positive CPR experience, yet do not want future classes to do CPR
21
Overcoming Student Resistance to a Teaching Innovation
Table 9
Timetable Summary Demonstrating Reflective Practice in Action
Semester
Student Criticism
Addressed
Instructor Actions
Major Outcomes
•Lack of instruction (9%N)*
•Total grading issues
(20%N)
•Did not enhance learning
(13%N)
Fall 2002
•Lack of instruction
•Grading complaints
•Prepared 4 page handout including
extensive grading explanation
Spring
2003
•Lack of instruction (2%N)
•Total grading issues
(13%N)
•Enhanced learning (5%P)
•Grading complaints
•Enhanced learning
•Proactively changed student grades
when deserved
•Began writing assignments
Spring
2004
•General grading issues
(0%N)
•Enhanced learning (7%P)
•Writing not fit for subject
(20%N)
•Writing not fit for
subject
Fall 2004
•Stated class was "writing-intensive"
•Increased CPR's worth to an exam
grade
•Writing not fit for subject
(23%N)
•Helped bad test takers
(5%P)
•Enhanced learning (10%P)
•Emphasized teaching philosophy,
47% of grade is not from exams,
and CPR is a grade in its own right
•General grading issues
(16%N)
•Writing not fit for subject
(20%N)
•Helped bad test takers
(10%P)
•Enhanced learning (23%P)
•Emphasized students are novice
reviewers; essays would be
regraded when asked
•Began to review student papers
before submission when asked
•Used class time to demonstrate CPR
and show importance of
writing/critiquing for future with
•General grading issues
(8%N)
•Writing not fit for subject
(8%N)
•Helped bad test takers
(4%P)
•Enhanced learning (18%P)
•Writing not fit for
subject
Spring
2005
•Writing not fit for
subject
•Grading Issues
Fall 2005
https://doi.org/10.20429/ijsotl.2008.020105
22
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Bloom's Taxonomy
•Grading Issues
•Used class time to demonstrate
how to critique
Spring
2006
•General grading issues
(0%N)
•Writing not fit for subject
(4%N)
•Helped bad test takers
(7%P)
•Enhanced learning (19%P)
* %N is the percentage of negative statements; %P is the percentage of positive statements.
Percent of Students
Who Agree or Strongly Agree
Graph 1
Student Assessment of Learning Gains (SALG) Results on How Students View CPR
80
70
Fall 2002
60
Spring 2003
50
Spring 2004
40
Fall 2004
30
Spring 2005
20
Fall 2005
Spring 2006
10
0
Others Should
Do CPR
CPR is
Enjoyable
Helps us Learn
Chemistry
Helps Improve
W riting
Helps Improve
Critiquing
Survey Questions
https://doi.org/10.20429/ijsotl.2008.020105
23
Overcoming Student Resistance to a Teaching Innovation
Graph 2
Odds Ratios Derived from Chi-Square 2x2 Contingency Tables Comparing Benefits to Students who
Enjoyed CPR to Those Students who Did not Enjoy CPR
25
Odds Ratios
20
Fall 2002
Spring 2003
15
Spring 2004
Fall 2004
10
Spring 2005
Fall 2005
5
Spring 2006
0
Helps us Learn
Chemistry
Helps Improve
Writing
Helps Improve
Critiquing
Surv e y Que stions
https://doi.org/10.20429/ijsotl.2008.020105
24
IJ-SoTL, Vol. 2 [2008], No. 1, Art. 5
Graph 3
The Relationship Over Time Between Student Experience with CPR and Their Promotion of CPR for
Future Classes
Percent of Students
60.0
50.0
Group 1 (%)
40.0
Group 2 (%)
30.0
Group 3 (%)
20.0
Group 4 (%)
10.0
0.0
Fall Spring Spring Fall Spring Fall Spring
2002 2003 2004 2004 2005 2005 2006
Semester
Group
Group
Group
Group
1:
2:
3:
4:
Students with negative CPR experience, yet want future classes to do CPR
Students with negative CPR experience, and do not want future classes to do CPR
Students with positive CPR experience, and want future classes to do CPR
Students with positive CPR experience, yet do not want future classes to do CPR
https://doi.org/10.20429/ijsotl.2008.020105
25
Overcoming Student Resistance to a Teaching Innovation
Graph 4
The Relationship Over Time by Group Between Student Experience with CPR Determined
Quantitatively and Whether or Not They Promote CPR for Future Classes and Their Percentage of
Positive Statement
Percentage
100
80
60
Group 1 (%)
40
Positive Statements (%)
20
0
Fall
2002
Spring Spring Fall
2003 2004 2004
Spring Fall
2005 2005
Spring
2006
Percentage
100
80
60
Group 2 (%)
40
Positive Statements (%)
20
0
Fall
2002
Spring Spring Fall
2003 2004 2004
Spring Fall
2005 2005
Spring
2006
Percentage
100
80
60
Group 3 (%)
40
Positive Statements (%)
20
0
Fall
2002
Spring Spring Fall
2003 2004 2004
Spring Fall
2005 2005
Spring
2006
Percentage
100
80
60
Group 4 (%)
40
Positive Statements (%)
20
0
Fall
2002
Group
Group
Group
Group
1:
2:
3:
4:
Spring
2003
Spring
2004
Fall
2004
Spring
2005
Fall
2005
Students with negative CPR experience, yet want future classes to do CPR
Students with negative CPR experience, and do not want future classes to do CPR
Students with positive CPR experience, and want future classes to do CPR
Students with positive CPR experience, yet do not want future classes to do CPR
https://doi.org/10.20429/ijsotl.2008.020105
26