STUDENT UNDERSTANDING OF CONSERVATION OF ENERGY AND MASS
I N INTRODUCTORY UNIVERSITY SCIENCE COURSES
BY
Jessica L. Odell
B.S. Eckerd College, 1997
A TH ESlS
Submitted in Partial Fulfillment of the
Requirements for the Degree of
Master of Science in Teaching
The Graduate School
The University of Maine
August, 2005
Advisory Committee:
Michael C. Wittmann, Assistant Professor of Physics, Advisor
Fra nqois G.Amar, Associate Professor of Physical Chemistry
Stephen A. Norton, Professor of Earth Sciences
STUDENT UNDERSTANDING OF CONSERVATION OF ENERGY AND MASS
I N INTRODUCTORY UNIVERSITY SCIENCE COURSES
By Jessica L. Odell
Thesis Advisor: Dr. Michael C. Wittmann
An Abstract of the Thesis Presented
in Partial Fulfillment of the Requirements for the
Degree of Master of Science in Teaching
August, 2005
In the Fall of 2004, student understanding of conservation of energy and
mass was measured in four introductory-level science courses (biology,
chemistry, earth science, and physics) at the University of Maine. Each course
fulfilled one semester of the University's general science education requirement.
A 20 question, multiple-choice survey was administered t o students in the four
courses, in a prelpost-test format. Ten questions on the survey involved the
application of the concepts of conservation of energy and mass i n either local or
system-wide situations, and were scored t o calculate gain.
Sub-groups of students were compiled by taking only those who were
taking one science course during the semester. Average normalized gain was
calculated for each sub-group t o allow for comparison between courses.
Students taking the biology course had significant improvement in the systems
applications, while students taking the chemistry course showed improvenient
on the local-level applications. Students enrolled concurrently in biology and
chemistry showed significant gains in both subsets of the survey, w i t h an overall
gain greater than students enrolled in each of the courses individually. Students
enrolled in the physics course showed no significant gains, while earth science
students showed significant negative gain on the local applications subset of the
survey. The results suggest that there is a difference between the introductory
courses that fulfill the University of Maine's general science education
requirement, in terms of improving student understanding of conservation of
energy and mass.
ACKNOWLEDGEMENTS
I thank, first and foreniost, my entire thesis committee, Drs. Michael
Wittmann, Fra nqois Aniar, Stephen Norton, and Mary Tyler, for their support
through this project. I also thank Dr. Susan Mcl<ay for her encouragenient and
for providing the opportunity t o pursue this degree.
In addition, I give niy gratitude t o the rest of the M.S.T. graduate
students wlio have made my experience quite enjoyable, especially David Nelson
who put up with my odd tastes in music all year.
Special thanks go t o my friends and family who have always supported
me in whatever endeavor I have undertaken.
TABLE OF CONTENTS
..
ACI<NOWLEDGEMEIVTS............................................................ II
LIST OF TABLES ............................................................................v
Chapter
I.
INTRODUCTION ......................................................................
Goals of General Science Education .......................................
Education Reform ..............................................................
Curriculum Development ..................................................
Learning Styles and Gender Issues.........................................
Learning Conservation of Energy and Mass .............................
Diagnostic Tests t o Assess Conceptual Understanding ...............
Development of Research Questions ......................................
r . SURVEY DESIGN AND ANALYSIS ...............................................
Survey Design ...................................................................
Survey Implementation .......................................................
Survey Analysis ..................................................................
3. RESULTS................................................................................
Overview ..........................................................................
Subset Analysis by Su b-Croup ................................................
Item Analysis by Sub-Group ..................
.
............................
Summary ..........................................................................
4. DISCUSSION...........................................................................
REFERENCES..............................................................................
APPENDI.CES...............................................................................
Appendix A . Free Response Survey ........................................
Appendix B . Pre-Test Survey ................................................
iii
Appendix C . Post-Test Survey ............................................... 3 8
Appendix D . Item Analysis ................................................... 41
Appendix E . Human Subjects Research Proposal ......................
42
BIOGRAPHY OF THE AUTHOR ...................................................... 47
LIST OF TABLES
Table I : Overall Score .................................................................................................
17
Table
2:
"Local" Subset ...............................................................................................
17
Table 3: "Systems" Subset ..........................................................................................
17
Table 4:Biology Students, First Semester, Non-Science Majors ........................... 18
Table 5: Biology Students, All Majors .......................................................................18
Table 6:Chemistry Students, All Majors..................................................................19
Table 7 : Biology and Chemistry, All Majors.............................................................19
.
Table 8:Farth Sciences Students All MaJats.................................................. 20
Table 9 : Physics Students, All Majors ......................................................................21
CHAPTER
I
INTRODUCTION
Two of the most important concepts in science today are the
conservation of energy, and the conservation of mass. These concepts serve as a
foundation for today's knowledge of science and natural phenomena and help us
understand the world around us. I designed a survey t o investigate student
understanding of these concepts in four different introductory-level science
classes at the University of Maine, Orono. Each of the four classes was i n a
different subject (biology, chemistry, earth science, and physics), and fulfilled
one semester of the University's general education science requirement. The
results of the survey showed that, although there was some small significant
gain in some areas, students are not learning the concepts of energy and mass
conservation as we would hope.
Goals of General Science Education
Most students at the University of Maine are required t o take at least t w o
semesters of science with a laboratory component in partial fulfillment of the
University's general education requirement. Courses can be in either the
biological or physical sciences (or both), and can be a basic or an applied science.
There are many classes that fulfill the requirement, and students are free t o take
whichever course they want, unless bound by specific degree or departmental
requirements. These core requirements are taken at any time before graduation,
unless they serve as pre-requisites for other degree-required courses.
Scientific literacy has been recognized as an important facet of education.
However, at the University of Maine, the reasons behind a general education
science requirement remain somewhat vague. Students often do not know why,
outside of "because i t is a general education requirement", they must take t w o
semesters of science i f they are majoring in a non-scientific field. Those students
with little direction, or who may not be comfortable with science, often opt for
an introductory, 100-level course that they hear is an "easy A". Many students
simply do not understand that the introductory courses in certain departments
may not be designed as "general education" courses, and are intended for
students majoring in the field. Thus, students taking the course only t o fulfill
the general education requirement often lack the sl<iIls necessary for success,
and are thus often unprepared for the work required in their chosen course.
The learning results for the University of Maine's general education
science requirements focus on t w o areas (Reports from General Education
Assessment Worlting Groups: Fall 2003, 2003). The first is general scientific
literacy. I t is expected that after completion of the requirements that students
can read and understand articles published in magazines such as "Science News".
The second result focuses on actual scientific knowledge. Students are expected
t o be able t o "demonstrate proper application of scientific principles"; however,
there is no specific guidance about teaching certain principles that are applicable
t o all sciences in any consistent fashion. Instead, each course can have its own
goals for achieving this second learning result, and its own methods o f
assessment (Reports from General Education Assessment Worlting Groups: Fall
2003, 2003).
This gives the individual courses and departnients the freedom t o
tailor their general education courses t o their specific needs. On the surface i t
does not suggest consistency among the courses from the viewpoint of the
courses being a part of the same general education requirement. Nor is there
always a distinction made between courses in a subject that fulfill a core science
requirement for non-science majors and courses that serve as a full i~itroductory
course for majors in that field. Some departments, such as Physics and
Astronomy do offer different introductory level courses for majors and nonmajors, both of which can fulfill the general science education requirement.
Biological Sciences, however, only offers one introductory-level course (BlOioo)
that car1 be taken by niajors and non-majors t o fulfill a general science
requirement.
Education Reform
There have been many recent efforts made t o improve the quality of
undergraduate science education (McCray, DeHaan, €4 Schuclc, 2003;
Reinventing undergraduate education: A blueprint for America 3 research
universities., 1998). The National Research Council (1999) published a report
that contained several initiatives and visions for improving undergraduate
science education. I t was suggested that the I<-12standards be brought forward
into the colleges as a part of admission standards, and that all students should be
required t o study science, mathematics, and technology at the University level.
As well, i t was noted that there are differences between students who take a
course as an introductory class in their intended major and students who are
learning science as a general requirement. The report suggested that students
who are learning science for different purposes should learn how t o learn for
their best advantage, and noted the difficulties in creating a course that
provided accessible and useful scientific learning t o all students (Transforming
Undergraduate Education in Science, Mathematics, Engineering, and
Technology, 1999).
Even before the initiatives on the national level, research had been done
in almost every field of science in an attempt t o discover how students learn
specific topics and what types of curricula work best. Instead of generalizing
approaches t o improving education, science educational research focuses on not
only the best way t o teach a student, but how t o improve science learning. The
material itself is the focus, with the goal being that students leave a class
actually understanding the concepts presented (McDermott,
2001).
Curriculum Development
Inquiry-based classes and laboratories are at the forefront of this
revolution, as they appear t o have a positive impact on student understanding
(McDermott,
2001)
(Willden, Crowther, Gu banich, & Cannon,
2002).
Inquiry-
based curricula involve students in the learning process more than traditional
lecture-style formats, and are starting t o be used in all types of science classes.
Sometimes just a few laboratory activities in an otherwise lecture-based class are
inquiry-based. However, more and more classes are attempting t o develop a
totally inquiry-based curriculum. Although there has been success w i t h
programs such as Physics By Inquiry (McDermott, 1996) and Worl<shop Physics
(Laws,
2004)) this
is often a daunting task for large, traditionally lecture-based
introductory university science classes.
Learning Styles and Gender Issues
In addition t o the teaching style, learning styles and gender issues are also
at the forefront of science education reform, as educators are trying t o achieve
equity in learning between the sexes. Researchers are trying t o determine if
differences in teaching styles can lessen the apparent gap in mathematics and
science sl<ills between men and women. The trend towards more inquiry-based
curricula may actually have a deleterious effect on gender equity, widening tlie
performance gap between males and females (Von Seclter & Lissitz, 1999).
The cause of this disparity is not fully understood, although there appears
t o be a gender-related difference in learning styles that, in the traditional
classroom situation, puts females at a disadvantage (Mayberry, 1998; Wee,
Baaquie, €4 Huan, 1993). This is not t o say that women are not just as capable as
men at learning science, but that the traditional learning environment appears
t o be better suited t o a more masculine learning style. Gender equity goes
beyond differences based purely on biological sex, and focuses on more
masculine or feminine learning styles. The differences in learning styles that are
not completely linked t o gender can make approaching gender equity difficult.
There has also been noted resistance t o making efforts t o use "gender-inclusive
pedagogy" in some instances (Rennie, 1998). I t is important t o note, however,
that gender equity is still a relevant area in science education research if the goal
is truly t o ensure that all students have a similar opportunity t o gain an
understanding of science from any science course.
Learning Conservation of Energy and Mass
In addition t o general studies on how t o improve the understanding of
science concepts through research on learning styles and teaching methods,
research has been done on student understanding of the specific concepts
themselves. The focus of my study is on student understanding of conservation
of mass and energy, concepts fundamentally integrated in thermodynamics. In
thermodynamics, a fair proportion of these studies have focused on conceptual
understanding and student misconceptions in chemistry and physics situations.
A basic premise of this research is that students enter the classroom w i t h their
own ideas about these concepts (Tytler, 2002).
Within chemistry, there is a strong emphasis on chemical equilibrium and
equation balancing in introductory-level thermodynamics. Students who are
proficient a t solving or balancing chemical equations seem t o have a good
understanding of the application of conservation of mass. They understand that
they need t o conserve "symbols, elements, or particles", yet researchers found
that conceptual understanding did not appear t o go beyond mechanical
manipulation (Yarroch, 1985).
Students have many misconceptions about the energy involved i n
chemical situations. These misconceptions run the gamut from chemical
equilibrium t o entropy, t o the first and second laws of thermodynamics (Thomas
& Schwenz, 1998). They are not limited t o students, as there is evidence that
secondary-level teachers still hold misconceptions in many, i f not all, of these
same areas (Banerjee, 1991, 1995). The prevalence of ideas such as the release of
energy through fire or burning (Barker & Millar, zooo), presents problems for
instructors when approaching concepts revolving around the underlying law of
conservation of energy.
Education researchers in physics have emphasized concepts related t o
mass and energy. Some of the hardest concepts for students t o grasp are those
of worlc and energy (O'Brien Pride, Volcos, & McDermott, 1998)) and i t has been
suggested that much of the confusion and many of the misconceptions arise
from the terms having many different definitions (Arons, 1999; Driver &
Warrington, 1985). Misconceptions about energy have a detrimental effect on
understanding conservation (Solomon, 1985). As well, good understanding and
application of thermodynamics concepts rely on scientifically correct notions of
work and heat (van Roon, van Sprang, €4 Verdonl<, 1994).
Today i t is rarely questioned that students have persisting misconceptions
about the ideas involved in thermodynamics. Some research has been done in
an effort t o discover the source of these misconceptions. First of all,
thermodynamics involves situations with multiple variables. Events w i t h more
than one or t w o variables are difficult for students t o understand, and there are
often ways of reducing the number of variables used t o describe a situation, or
t o solve a particular problem. The perpetuation of these reductions can lead t o
generalizations that prevent conceptual understanding (Rozier & Viennot, 1991).
There is also evidence that the historical development of the field of
thermodynamics can have an effect on persistent misconceptions. The language
used today is the same that was used in the past t o describe or explain
phenomena as the understanding of thermodynamics was developing. The
terms and definitions may not be tlie best representatives of today's body of
Itnowledge, and have been found t o reinforce misconceptions (Cotignola,
Bordogna, Punte, & Cappannini,
2002).
As in the field of chemistry, physics has
its own share of teacher-based misconceptions. Elementary school teachers are
responsible for many students' first foray into scientific concepts, yet Lawrenz
found that many of them do not liave a good understanding of basic physical
science concepts (Lawrenz, 1986). Bauman found that physics textbooks also
contain corrlmon errors. Many concepts are not presented clearly in the texts,
which perpetuates rr~isconceptionsand/or leads t o confusion about topics such
as energy and thermodynamics (Bauman, 1992). These issues w i t h conceptual
understanding are not limited t o physics, and can be found in the biological
sciences as well (Cho, I<ahle, & Norland, 1985).
Diagnostic Tests to Assess Conceptual Understanding
In the next chapter, 1 describe the development of the multiple-choice
test used t o assess student understanding of conservation of energy and mass.
Assessing conceptual understanding has been a challenge t o researchers.
Although interviews can give deeper insight into exactly how a student is
processing Icnowledge in order t o solve a problem, they are difficult t o
implement in a large-scale study. Multiple-choice tests are much easier t o
implement, and thanks t o computers, much easier t o score for analysis.
Conceptual inventories in a multiple-choice format have been developed in
many areas t o test students' understanding of basic scientific concepts, and t o
identify misconceptions (Nazario, Burrowes, & Rodriguez, 2002). In physics,
common tests include the Force Concept Inventory (FCI) (Hestenes, Wells, &
Swackhamer, 1992)~and the Force and Motion Conceptual Evaluation (FMCE)
(Thornton & Sol<oloff, 1998). Both of these diagnostics have been used w i t h
success in physics classes across the country. Supplementary studies have shown
that there is some correlation between scores on the FCI and scores on more
traditional written examination problems that focus on the same areas.
However, the correlation is not as strong as one would hope, as typical written
problems often do not rely on or test for conceptual understanding (Stein berg &
Sabella, 1997).
Multiple-choice concept inventories are also being developed for use in
other areas of science. The Journal of Chemical Education Online has been
collecting and developing inventory questions for the various subject areas in
chemistry (JCE QBank, 2003). Similar inventories are being developed for the
biological sciences, including one for natural selection. Natural selection is a
difficult concept in biology, and is filled with misconceptions and alternate
conceptions. The inventory, a 20-question multiple-choice test, was shown t o
be a valid and useful tool for this topic (Anderson, Fisher, & Norman,
2002).
Development of Research Questions
A t the University of Maine students may choose from a wide-range of
courses in order t o fulfill the general "science" requirement. -The very broad
guidelines for the general science education requirement described previously
allow for ambiguity as t o the equality of science instruction in the different
departments.
Conservation of Energy and Mass are concepts that are applicable t o all
sciences, and can be found in almost every science textbook. Student
understanding of concepts surrounding energy have been studied fairly
extensively in the recent past, mostly in physics- and chemistry-based situations.
Studies are often directed t o very specific situations, such as how students
perceive and use energy ideas t o solve certain types of mathematical equations
in chemistry or physics.
While Chemistry courses often include direct instruction on conservation
of energy problems (usually done mathematically), other science courses skim
over this concept, if they mention i t at all. Biology classes typically do not go
into detail about conservation of energy or mass, although i t is commonly in the
background of many topics, such as metabolism and the water cycle. Biology
topics that use conservation of energy or mass are typically a t a systems level,
while chemistry ideas normally deal w i t h local descriptions.
In order t o gain some preliminary insight into the general science
education requirement at the University of Maine, I posed the following
questions:
I. Do general education students in introductory level science
courses learn Conservation of Energy and Mass?
2.
Are there differences in student understanding of Conservation
of Energy and Mass between introductory science courses that fulfill the
general science education requirement?
The intended study population was non-science majors taking only one
science course and in their first semester at the University of Maine. In addition
t o the main research questions, I also wanted t o see i f there was any correlation
between a student's conceptual understanding of energy itself (how they
defined "energy") and their understanding of conservation, as well as any
correlations between attitudes about conservation and understanding.
CHAPTER
2
SURVEY DESIGN AND ANALYSIS
I developed a 20-question survey, based on preliminary results from a
short questionnaire administered t o summer students at the University of
Maine, t o test student understanding of and attitudes toward the concept of
conservation of energy and mass. The survey was administered during the Fall
semester of 2004 t o students in four introductory science classes, as both a pretest and post-test. Four subject-specific sub-groups within the full set of results
were created by isolating students taking only one science class. Results from
each sub-group, and the entire cohort of students were analyzed using average
normalized gain (<g>).
Survey Design
During the summer of 2004, students attending science classes at the
University of Maine were asked t o fill out a short (6 questions) free-response
survey about the concepts of energy and open-and closed systems (Appendix A).
The questions asked for the students' personal definitions o f energy, open and
closed systems, and "conservation of energy". In addition, students were asked i f
they agreed or disagreed with t w o different statements: one about plants
making their own energy, and one about the applicability of the first law of
thermodynamics t o biological systems. Students were also asked t o explain their
reasonings on the t w o multiple-choice questions.
The format of the survey allowed students t o feel free t o use whatever
language w i t h which they were comfortable. Common student responses were
used in part t o generate answer choices t o the final multiple-choice questions
about the students' understanding of energy and energy systems. This allowed
for these final answer choices t o be written in language the students typically
used, rather than in formats typically seen in textbool<s, which could bias the
responses.
The final survey (Appendix B) was designed with three sections. The first
section included questions about the students' major, semester in school, other
science courses currently being talten, and previous science classes talten. These
questions were designed t o allow for parsing the data set into sub-groups, such
as only non-science majors, or students in their first semester at University.
Because information in each class could easily be classified, it was possible t o ask
these questions in multiple-choice format. This insured consistency with the
entire survey and allowed for quicker analysis of data.
The second section consisted of twelve questions designed t o evaluate the
students' understanding of energy and systems, as well as their Itnowledge of
situations involving the conservation of mass and energy. The first t w o
questions in this section were designed t o categorize how students thought
about and defined energy and energy systems.
6. Energy can best be described as?
A: a force needed t o do worl<
B: heat
C: work done on an object
D: an ability t o do work
E: an interaction between molecules
7. In closed systems energy is
, and in open systems energy is
A: stored t o be used at a later time; cannot be stored for later use
B: stays in the system; doesn't stay in the system
C: limited; unlimited
D: conserved; isn't conserved
E: recycled; lost
As there could be more than one possible "correct" answer, these questions were
not included in the final percentage score and do not figure in the normalized
gain. Answer choices for these t w o questions were created based in part on the
types of responses gained from summer students. -They were designed
specifically t o represent broad categories of possible ideas in language with
which students themselves are comfortable and t o intentionally avoid
mimicking definitions or phrases that could be seen in textbooks. The goal was
t o create options that used similar and non-scientific language, so as t o avoid as
much bias as possible.
The remaining ten questions were designed t o test the students'
understanding of applied forms of energy and mass conservation as related t o
physics, chemistry, biology, and earth science. The questions were split evenly
between those relating t o local, small-scale energy situations, and those relating
t o the larger, systeni-based energy situations. These ten questions all had single
correct answers, and were the scored section of the questionnaire.
The questions for the "local" subset focused around situations related t o
chemistry and physics. Originally, i t was hoped t o focus questions around
specific situations presented in the courses. However, only the chemistry class
presented the subject of conservation of mass or energy in any specific context.
In light of this and t o attempt t o avoid class-specific bias, questions were
formatted in a more general context. Calculations beyond very basic addition or
subtraction were also left out of all but t w o questions (8 and 14, see Appendix
13). This was done t o ensure that the survey was testing more conceptual
knowledge and not biased against those students w i t h limited mathematical
abilities. The chemistry questions (questions 8-11) were adapted from
conceptual questions on energy and mass conservation found in the "QBank" of
the Journal of Chemical Education's website (JCE QBank, 2003) w i t h the help of
committee members Drs. Wittmann, Amar, and Norton. The final question in
the "local" subset (question 14) was written t o reflect a typical momentum
situation found in many physics classes.
The five "system" questions were related t o situations found in geology
(questions 15-17) and biology (questions
12 and
13). Unlike the questions in the
'local" subset, these questions were broader in scope. They focused on how (or
if) conservation applied t o biological systems, and geological phenomena such as
avalanches. Final forniats of these questions were arrived a t w i t h the help of all
con-~niitteemembers and Earth Science graduate student Eric Rickert.
The final section of the survey consisted of three Likert-scale type
questions (questions 18-20). Students were asked how much they agreed or
disagreed with statements about how conservation ideas applied t o the course
they were specifically taking, the field in general, and how the idea would be
presented in the course. The post-test survey (Appendix C) asked the same
questions, but in the past tense.
Survey Irr~plementation
Four introductory, ioo-level science classes at The University of Maine in
the Fall 2004 term served as the target audience for this survey: BlOioo,
CHYiri, PHYios, and ERSioi. The biology (BlOioo) course was chosen as i t is
the only introductory course offered in the department. There are 3
introductory earth science courses, and ERSioi was chosen as the lowest-level
introductory course w i t h a concurrent laboratory. Since the laboratory is
required in BIOioo, I decided t o choose courses in all departments with a
laboratory component, if possible. CHYi2.1, like ERSioi, is the lowest-level
introductory course with a concurrent laboratory section in the Department of
Chemistry. PHYio5 is a single-semester introductory course (as compared t o
introductory courses that are a part of a two-semester sequence) that was
designed as a general education course for non-majors. During the semester
studied, PHYios was taught as part of a reform project headed by Michael C.
Wittmann, funded in part by NSF grant DUEo4-10895. The final criterion for
choosing courses was focused on non-science majors. As one facet of the study
was t o investigate the learning due t o the core general science requirement, I
chose t o focus on the courses that would not contain primarily majors within
that field, but hopefully a wide range of students.
The pre-test version of the survey and informed consent form were
handed out in each class during the first 3 weeks of the Fall term, so as best t o f i t
in with scheduling. Students were asked t o sign the consent form and fill out a
bubble-sheet w i t h their answers only if they wished t o participate in the study.
They were also verbally informed that their answers would remain anonymous
and performance would not affect their grade in the course. Students unwilling
t o participate, and those under the age of 18, were simply asked t o hand back
their questionnaire and blank bubble sheet uporl leaving the classroom. The
post-test version of the questionnaire was admil-~isteredin a similar fashion
during the last 3 weeks of the semester, so as not t o conflict w i t h final exams.
Survey Analysis
Both the pre-test and the post-test versions of the questionnaire were
scored at the University of Maine's computer testing center. Responses t o each
question were recorded, along with a percentage score for questions 8-17, into
an Excel spreadsheet. This allowed for efficient data correction, as the scanners
were not able t o record multiple answers, which were acceptable for certain
questions. Missing values and unreadable responses were double-checked w i t h
the students' bubble-sheets. Once the data were corrected, each student was
assigned a distinct identification number t o keep responses confidential. Posttest responses were scored in the same fashion. Missing information and
unreadable responses were corrected by referencing the bu bble-sheets, and ID
numbers were assigned by using the key from the pre-tests.
Preliminary data reduction was done in several ways. First, only students
who had consented t o having their responses used in the study, and had
completed both the pre-and post-test versions of the survey, were included in
the data set. There were few ways t o determine how seriously students were
answering questions. In light of this, the 3 true-false questions were used as a
benchmark. "A" and "B" responses were the only acceptable answers for these
questions, so students answering "C", "D", or "E" t o any of the questions were
excluded. This preliminary reduction brought the data set t o a total of 541
records. For each student, percentage scores were calculated for the subsets of
"local" and "system" questions for both the pre-and post-tests.
Average normalized gain, <g >, was calculated for the overall data set for
the scored questions in 3 parts: the entire set, the "local" set, and the "systems"
set. These <g > values served as the baseline comparison for the su b-groups.
< g > = (Average Pre-Test - Average Post-Test)/(? 00 - Average Pre-Test)
Sub-groups were initially proposed based on semester in school, major, and
concurrent courses. Originally, sub-groups for each course were planned t o be
restricted t o those students in their first semester of college, non-science majors,
and only those taking one science class (the one in which they filled out the
survey). As some of the courses surveyed were smaller than others, i t was
necessary t o broaden the scope for cross-class com parisons. The final subgroups
were Biology, Chemistry, Earth Science, Physics, and combined BiologyChemistry students.
The cross-course comparisons were done with students taking no
concurrent science classes, but major and semester in college were not limited.
The same 3 <g>-values were calculated for each of the sub-groups as for the
entire data set. In addition t o the calculations of <g>-values, a Student's T-Test
was run (two-tailed, alpha=o.og) for overall gain, local gain, and systems gain
for the overall data set, and each sub-group.
CHAPTER 3
RESULTS
The pre-test results were analyzed using ANOVA t o check for any effect
of semester in school, course, and major on the pre-test score. No significant
effects were found in any of these areas, suggesting that all students were
starting the semester with a similar understanding of conservation o f energy
and mass. These early results allowed me t o compare average normalized gains
across courses.
Overview
-The full set of students had a small, yet significant gain on the overall
survey (Table I). There was not a significant gain on the "local" subset (Table 2),
so most of the overall gain was due t o the improvement on the "systems" subset
of the survey (Table 3). Because BlOioo contributed the majority (48%) of the
students t o the overall data set, biology-related effects could have biased the
results for the group of all students.
Sub-groups from each course were limited t o students taking only that
specific science course during the Fall 2004 semester. The su b-groups from the
chemistry and physics classes did not show significant improvement on the
overall survey. However, both of these sub-groups had small sample sizes (n=
18 and 14, respectively). The Earth Science sub-group showed a significant
decrease in overall score, with a < g > of -0.18 (p=o.o2). The Biology sub-group
did, however, show a significant gain in overall score: <g > =
0.11,
p=
0.00.
On the "local" subset of the survey, 3 of the sub-groups had no significant
gain. The Earth Science sub-group had a significant negative gain. -The
Chemistry sub-group had a 4g > of 0.30, yet it was not significant, most Ii kely
due t o the small sample size. The "systems" subset showed results similar t o
those of the overall survey, with significant gains seen in the Biology sub-group.
Su b-Group
% pretest
% posttest
<g>
p-va lue
n
All Students
51-91
58.87
0.14
0.00
541
Biology
50.46
56.03
0.11
0.00
219
Chemistry
61.11
65.56
0.11
0.32
I8
Earth Science
63-41
57.05
-0.17
0.09
44
Physics
52.14
52.86
0.01
0.92
14
Biology/Chemistry
49.21
61.52
0.24
0.00
165
Table I: Overall Score
Sub-Group
% pretest
% posttest
<g>
p-va lue
n
All Students
61.59
63.29
0.04
0.17
541
Biology
59.18
59.63
0.01
0.80
219
Chemistry
70.00
78.89
0.30
0.07
I8
Earth Science
68.18
54.55
-0.43
0.02
44
Physics
57.14
58.57
0.03
0.88
14
Biology/Chemistry
60.73
67.64
0.18
< 0.01
165
Table
2:
"Local" Subset
Sub-Group
YO pretest
% posttest
<g>
p-va lue
n
All Students
42.23
54.35
0.21
0.00
541
Biology
38.26
52.42
0.23
0.00
219
Chemistry
58.89
52.22
-0.16
0.16
18
Earth Science
60.91
59.55
-0.03
0.70
44
Physics
50.00
47.14
-0.06
0.69
14
Biology/Chemistry
39.27
55.39
0.27
0.00
165
Table 3: "Systems" Subset
17
Subset Analysis by Sub-Groups
The original sub-group of interest was non-science majors in their first
semester at University, taking only one science class. Only 75 o f the matched
541 students fulfilled this requirement. Of these, only the BlOioo class, w i t h 62
students, had a large enough population t o be studied (Table 4).
cluster
% pretest
% posttest
<g>
p-va lue
overall
48.71
54.52
0.11
0.05
local
57.10
58.06
0.02
0.78
systems
40.00
50.97
0.18
<O.OI
Table 4: Biology Students, First Semester, Non-Science Majors. (n=62)
The gains in this sub-group from the BlOioo class (Table 4) were similar
t o those of all students, with the largest gains on the "systems" subset. Due t o
smaller sample sizes in the other courses, these limitations on the data proved t o
be too specific for cross-class comparison. When the sub-group of all Biology
students was tested for effects of major and semester in school, no significant
effects were seen. In light of this, all course-specific su b-groups were limited
only by "concurrent course". Only students taking one science class were
included, regardless of major or semester in school.
The biology students (Table 5), like the group of all students, have
significant gain in the overall survey, as well as the "systems" subset. I t is lil<ely,
however, that the overall gain is due solely t o the gain on the "systems" subset.
cluster
% pretest
% posttest
<g>
p-value
overall
50.46
56.03
0.11
0.00
local
59.18
59.63
0.01
0.80
systems
38.26
52.42
0.23
0.00
Table 5: Biology Students, All Majors. (n=219)
I8
Chemistry students (Table 6) Iiad no significant gain on any subset of the
survey. However, this sub-group was fairly small (n = 18). The high average
normalized gain on the "local" subset, < g > = 0.30, suggests that there may be an
effect of the course on student understanding of conservation of energy in local
situations. This course is also the only one that teaches the idea of conservation
in an explicit fashion, typically mathematically.
cluster
% pretest
% posttest
<g>
p-value
overall
61.11
65.56
0.11
0.32
local
70.00
78.89
0.30
0.07
systems
58.89
52.22
-0.16
0.16
Table 6: Chemistry Students, All Majors. (n=18)
A common occurrence a t the University of Maine is for students t o take a
biology class at the same time as a chemistry class. These t w o courses,
individually, seemed t o have an effect on understanding in one of the t w o areas
of the study. A sub-group was compiled of students taking biology and
chemistry concurrently, with no other concurrent science classes that semester
(Table 7).
cluster
% pretest
% posttest
<g>
p-va lue
overall
49.21
61.52
0.24
0.00
local
60.73
67.64
0.18
<O.OI
systems
39.27
55.39
0.27
0.00
Table 7:Biology and Chemistry Students, All Majors. (n=165)
This sub-group of students shows significant improvement in the survey
overall, as well as each of the subsets of the survey. The <g >-values for the
Biology-Chemistry sub-group were higher in all instances than in either the
Biology or Chemistry sub-groups. Significant gains seen i n the Biology sub-
group in the overall survey and in the "systems" subset were most likely
reflected in the gain seen in the Biology-Chemistry sub-group. The BiologyChemistry subgroup had a 2% greater increase on the 'systems" subset, and a
6% greater increase on the overall survey, compared t o the Biology sub-group.
The "local" subset of the survey is where the greatest difference occurs.
lleither tlie Biology nor the Chemistry sub-groups showed a significant gain.
Although the Chemistry only su b-group was biased by a small sample size, the
group had a <g > value of 0.30, representing a 9% increase in score. The
Biology-Chemistry sub-group had a smaller <g>, only 0.18, but that represented
an 8% increase between pre- and post-test, compared t o less than a 1% increase
in the Biology sub-group. More iniportantly, when compared t o the group of all
students, there was a significant gain in the "local" subset of the survey.
The other t w o sub-groups, Earth Science and Physics, did not show
significant positive gains in any subsets of the survey.
Earth Science students (Table 8) showed significant negative gain on the
"local" subset of the survey. Although not significant, there were also negative
gains on the overall survey ( < g > = -0.17)~as well as on the "systems" subset
(<g > = -0.03).
cluster
% pretest
% posttest
<g>
p-value
overa l l
63.41
57.05
-0.17
0.09
local
68.18
54.55
-0.43
0.01
systems
60.91
59.55
-0.03
0.70
Table 8: Earth Science Students, All Majors. (n=++)
Reasons for the negative normalized gains are unl<nown and worthy of
further exploration. Below, I discuss some items on the survey, while I discuss
more general points here. I t may be that the course does not specifically focus
on conservation of energy and mass and may present these concepts in a way
20
that confuses students; I have no specific evidence for this, but suggest that i t be
investigated further. I t may also be that the negative normalized gains come
from the prevalence of non-science majors in the data set. Thirty-eight out of
the 44 students taking ERSioi as their only science course reported being nonscience majors in at least their third semester at the University of Maine. I t is
possible that these students were unprepared for the course, in terms of the
sl<iIls i t required.
There was no significa~itchange in scores for the Physics sub-group (Table
9). Part of this is Iiltely due t o the small sample size of only 14 students.
PHYio5, Ii ke many of the other courses, does not specifically teach the concepts
of conservation of energy and mass, therefore i t is not surprising t o see no
significant gain on any portion of the survey.
cluster
% pretest
% posttest
overall
52.14
local
systems
gain (<g>)
p-value
52.86
0.01
0.92
57.14
58.57
0.03
0.88
50.00
47.14
-0.06
0.69
Table 9: Physics Students, All Majors. (n=14)
Item Analysis by Sub-Group
In order t o investigate some of the significant gains in the sub-groups,
I calculated average normalized gain for each of the 10 scored questions on the
survey. The complete analysis for each question can be found in Appendix D.
I discuss several of these questions as they relate t o the individual sub-groups.
Question 17 (Appendix B) concerned the conservation of energy as i t
related t o the initial phase of an avalanche. Although an earth science question,
the underlying concepts of a shift from potential energy t o kinetic energy were
taught specifically in PHYio5. Only 50% of the students in the Physics subgroup answered this question correctly on the pre-test, while 93% answered
21
correctly on the post-test (<g> = 0.86). The Biology sub-group also showed
similar gains on this question, with an increase from 34% t o 73% correct. The
Biology-Chemistry sub-group had positive gains as well, although the Chemistry
sub-group had a slight decrease (5%) in score. The Earth Science sub-group also
had a slight negative gain on this question, although i t is a subject covered in the
course. The decrease from 79% t o 75% correct corresponded t o a difference of 2
fewer students answering correctly, and led t o a value of <g> = -0.19. Thus,
starting w i t h a high score and decreasing only a little can lead t o a large negative
gain value.
Question 16 (Appendix B) is another question related t o earth science. I t
asks students about the "water cycle", in terms of what is being conserved. The
correct answer t o this question is conservation of mass, yet conservation of
water was also accepted. Students over-all did poorly on this question, w i t h a
post-test average of 45%. Earth Science students, however, did not perform as
expected. Although they did improve (from 60% t o 64%), the shift only
equated t o 2 students changing their answers. As the water cycle is a part of
introductory earth science courses, i t is notable that these students did not have
a much larger gain.
Four of the five "local" application questions related directly t o chemical
reactions and situations. Two of these questions stand out, questions 8 and 10
(Appendix B).
The central concept of question 8 was the conservation of mass. Students
were asked the final mass when i k g of salt was dissolved in r o k g of water. The
correct answer is 21kg. The Chemistry sub-group showed the greatest gain
(<g > = 0.35). -This is expected, as the course teaches this concept quite
specifically. The Biology-Chemistry sub-group showed a gain of 11% (<g> =
0.29)~although Biology students had a small loss. What stands out in the
analysis of this question is the large negative gain (<g>=-0.80) in the Earth
Science su b-group. A t the start of the semester, 75% of the earth science
students correctly answered this question, dropping t o a low 55% at the end of
the course. Tlie remaining 45% of students thought that the final mass was
either 2okg (27%) or between 20 and xI<g (18%). Students answering this
question may be led astray by thinking about volunle rather than mass. Tlie
volume of the solution stays nearly unchanged. I t is possible that the Earth
Science sub-group includes students who developed a confusion between mass
and volume during the course of the semester. Further research may uncover
what led t o their decrease in correct scores.
Question
10 also
involved conservation of mass. Students were asked i f
mass is destroyed when a match is burned. A correct answer stated that the
mass is not destroyed, but that the atoms are rearranged. As with question 8,
gains were seen in the Chemistry and Biology-Chernistry sub-groups (<g > = 0.61
and <g> = 0.29, respectively) . Biology students also showed a gain (<g> =
0.21)~although this type of question was not taught in B101oo. Earth Science
students, however, showed a similar pattern t o question 8. A t the beginning of
the semester, 73% of the earth science students were able t o correctly say that
matter is not destroyed in tlie process of burning a match. However, at the end
of the semester, over 40% of students chose answers that specifically said that
mass was destroyed.
Two observations are clear from tlie item analysis. First, items do
measure class content appropriately. Though the overall and subset gains by
PHYio5 students were not significant, the one topic on the survey that was
taught in class showed a large gain. Similarly, cher~iistrytopics showed large
gains in the appropriate subgroups. Second, we find a possible explanatio~ifor
the Earth Science subgroup having significant negative nornialized gains on the
subset of local application questions. More detail is needed t o understand what
led them t o change their responses. I t may be that mass and volume confusions
arose during instruction. I can think of no reasons why more Earth Science
students would believe that mass can be destroyed.
Summary
Overall, these cross-course comparisons show that there is a difference in
the effect on student understanding of the concepts of conservation of mass and
energy a t the introductory course level
As the original sub-group of interest was only non-science majors in their
first semester a t university, the su b-groups were tested (ANOVA) for effect of
both major and semester. No significant effect was seen from major or semester
in each of the sub-groups. This is not surprising, as these are introductory
courses. Science majors will typically take these early in their University career,
suggesting little or no subject-specific background at the University level. Only
69 of the reported 245 science majors reported taking a prior science class.
Twelve of these were in their first semester of college, suggesting that some of
the reported "previous" courses were in fact at the secondary level. Non-majors
may take these courses at any time, yet i t is unlikely that they will have had any
other science courses a t the University level. Of the 277 non-science majors, 170
did not report having any prior University level science bacl<ground. Only eight
of the non-majors in their first semester at University reported prior science
education, again, probably suggesting courses taken at the secondary level.
Although there was some apparent confusion in answering the question
about previous courses, these courses did not have a significant effect on either
pre-test score or gain. I t seems that the introductory Biology course has the
greatest effect on student understanding of systems-level energy conservation,
even though this is a topic that is not covered in any depth. The addition of a
chemistry course not only increases the gain in understanding of local
application of conservation of energy and mass, but appears t o increase gain
overa l I.
The earth science course, like the others, also does not teach conservation
of energy and mass in any detail. However, the significant negative gains on the
local subset of the survey, and hence the survey overall, suggest that the
context of the course and the format of the questions on the survey were not
com pati ble.
CHAPTER 4
DISCUSSION
Based on the data, students taking both a biology course and a chemistry
course are niore lil<ely t o learn both local-level and systems-level applications of
conservation of energy and mass. I t seems that they learn the local-level
applications in the chemistry course, and the systems-level applications in the
biology course. There seem t o be positive non-linear effects of taking both
courses at once. Causes are ur~I<nown
and would require further testing, but the
2
classes together have the strongest effect on understanding of the
conservation of energy and mass in applied situations, even though i t is not
expressly taught in the classes.
The introductory earth science course, ERSioi, does not seem t o proniote
understanding of conservation of energy and mass in the same context as the
introductory biology course, BlOioo. Neither one of these courses teaches the
concepts in any specific format, and both are often taken by non-science majors.
Further study would have t o be done t o investigate the differences between the
courses that cause the wide differences in understanding of conservation of
energy and mass. The physics course in this study, PHYio5, also did not promote
a gain in understanding. However, this is a new course, designed specifically as a
general education physics course, and the other courses in the department (ones
typically taken by physics, engineering, and biological science majors) should be
investigated.
The item analysis shows that certain questions do measure course
contents appropriately, as seen in the results from the Chemistry and Physics
sub-groups. The large negative normalized gains seen in the Earth Science subgroup on many of the "local" applications questions suggest that more in-depth
research be done in order t o identify and rectify possible causes of the shift from
correct t o incorrect responses about conservation of energy and mass.
These preliminary results show that there is indeed a difference between
i~itroductoryscience classes that f u l f ~ lthe
l general science education
requirement at the University of Maine, with respect t o understanding of t w o
core science concepts. Much research has been done on how students learn
specific concepts, such as 'energy" or the laws of thermodynamics in isolated
chemistry or physics courses. There have also been directives t o improve science
education a t all levels, including the undergraduate level. Along w i t h
improvement, there have been efforts t o better define what is expected out of
science classes at this level. (Ben-Zvi, 1999).
The University of Maine's requirements for the core science courses are
fairly broad.
Read and comprehend articles in one or several areas of science
from sources at the level of Discover magazine or Science News.
Students should demonstrate proper application of scientific
principles. (Reports from General Education Assessment Working
Groups: Fall 2003, 2003)
The assessment for each course is left up t o the department and can take
one of many forms, including pre-and post-tests and portfolios. There is
nothing, however, that discusses concepts that should be learned in the science
classes that fulfill the general education requirement, nor assessments that show
comparability between courses. The results from this study show that, i n
regards t o conceptual understanding and application, general science education
courses do not provide similar experiences. If the goal of the general science
education requirement at the University of Maine is for students t o demonstrate
proper application of any scientific principle, then perhaps the general learning
results and broad assessment guidelines are sufficient. However, i f the goal is t o
provide similar learning experiences, regardless of the general education course
taken, the evidence suggests that the goal is not currently being met.
The importance of science education is rarely debated, and non-science
majors a t the University of Maine have t o fulfill a small science requirement. Of
the 295 students only taking one science course, 200 of them reported being
non-science majors. The earth science and physics courses had the highest
percentage of non-majors, both 869'0, and these t w o courses did not show gain
in any portion of the survey. In contrast, the Biology/Chemistry sub-group only
was o r ~ l y32% non-science majors. The best learning of t w o of the most
fundamental concepts in sciences is happening in courses taken by the fewest
non-science majors.
In this thesis, I created a research instrument that allowed me t o study
student understanding of the conservation of energy and mass. The evidence
from the survey suggests that non-science majors are not learning concepts so
basic that we assume they are known by all college students. By recognizing the
weakness of our assumptions, we can move forward t o help improve student
learning. These preliminary results can serve t o encourage reforms in the
studied courses, with the long-term goal of having all students at the University
of Maine fulfill both aspects of their general science education laboratory
science requirements.
REFERENCES
Anderson, D. L., Fisher, I<. M., & Norman, G. J. (loor). Development and
Evaluation of the Conceptual Inventory of Natural Selection. Journal o f
Research in Science Teaching, 39(10), 952-978.
Arons, A. B. (1999). Development of energy concepts in introductory physics
courses. American Journal o f Physics, 67(12), 1063-1067.
Banerjee, A. C. (1991). Misconceptions of students and teachers i n chemical
equilibrium. International Journal o f Science Education, 13(4), 487-494.
Banerjee, A. C. (1995). Teaching Chemical Equilibrium and Thermodynamics in
Undergraduate General Chemistry Classes. Journal o f Chemical Education,
72(10), 879-881.
Barker, V., & Millar, R.
(2000).
Students' reasoning about basic chemical
thermodynamics and chemical bonding: What changes occur during a
context-based post-16 chemistry course? lnternational Journal o f Science
Education, rr(ii),1171-1200.
Bauman, R. P. (1992). Physics the Textbook Writers Usually Get Wrong: II. Heat
and Energy. The Physics Teacher, 30, 353-356.
Ben-Zvi, R. (1999). Non-science oriented students and the second law of
thermodynamics. International Journal of Science Education, 21(12), 12514268.
Cho, H.-H., I<ahle, J. B., & Norland, F. H. (1985). An Investigation of High School
Biology Textboolts as Sources of Misconceptions and Difficulties in
Genetics and Some Suggestions for Teaching Genetics. Science Education,
69(5), 707-719.
Cotignola, M . I., Bordogna, C., Punte, C., & Cappannini, 0. M. (2002). Difficulties
in Learning Thermodynamic Concepts: Are They Linlted t o the Historical
Development of this Field? Science & Education,
77,
279-291.
Driver, R., & Warrington, L. (1985). Students' use of the principle o f energy
conservation in problem situations. Physics Education, 20,171-176.
Hestenes, D., Wells, M., & Swacl<hamer, G. (1992). Force concept inventory. The
Physics Teacher, 30(3), 141.
JCE QBank. (2003). Retrieved April,
2005,
frorr~~ , / / ~ c h e m e d . c h e m . w i s c . e d u
Lawrenz, F. (1986). Misconceptions of Physical Science Concepts Among
Elementary School Teachers. School Science and Mathematics, 86(8),
654-660.
Laws, P. W. (2004). Workshop Physics Activity Guide. New York: John Wiley €4
Sons.
Mayberry, M . (1998). Reproductive and Resistant Pedagogies: The Comparative
Roles of Collaborative Learning and Ferr~ir~ist
Pedagogy in Science
Education. Journal of Research in Science Teaching, 35(4), 443-459.
McCray, R. A., DeHaan, R. L., & Schuck, J. A. (Eds.). (2003). Improving
Undergraduate Instruction in Science, Technology, Engineering, and
Mathematics. Washington, D.C.: The National Academy Press.
McDermott, L. C. (1996). Physics by Inquiry. New Yorlc John Wiley & Sons.
McDermott, L. C.
(2001).
Oersted Medal Lecture 2001: "Physics Education
Research - The I<ey t o Student Learning". American Journal o f Physics,
69(11), 1127-1137.
Nazario, G. M., Burrowes, P. A., & Rodriguez, J.
(2002).
Persisting
Misconceptions: Using Pre- and Post-Tests t o Identify Biological
Misconceptions. Journal of College Science Teaching, 31(5), 292-296.
OIBrien Pride, T., Vol<os, S., & McDermott, L. C. (1998). The challenge of
matching learning assessments t o teaching goals: An example from the
work-energy and impulse-niomentum theorems. American Journal of
Physics, 66(2), 147-157.
Reinventing undergraduate education: A blueprint for America's research
universities. (1998). Menlo Park, CA: Carnegie Foundation for the
Advancement of Teaching.
Rennie, L. J. (1998). Gender Equity: Toward Clarification and a Research
Direction for Science Teacher Education. Journal o f Research in Science
Teaching, 35(8), 951-961.
Reports from General Education Assessment Worlting Groups: Fall 2003. (2003).
Orono, Maine: University of Maine.
Rozier, S., & Viennot, L. (1991). Students' reasonings in thermodynamics.
International Journal of Science Education,
13(2),
159-170,
Solomon, J. (1985). Teaching the Conservation of Energy. Physics Education,
20(4),
165-170.
Steinberg, R. I\., & Sabella, M . 5. (1997). Performance on Multiple-Choice
Diagnostics and Complementary Exam Problems. The Physics Teacher, 35,
150-155.
Thomas, P. L., & Schwenz, R. W. (1998). College Physical Chemistry Students'
Conceptions of EquiIi bri um and Fundamental Thermodynamics. Journal
o f Research in Science Teaching, 35(1 o),
1151-1160.
Thornton, R. K., & Sokoloff, D. R. (1998). Assessing student learning o f Newton's
laws: The Force and Motion Conceptual Evaluation and the Evaluation of
Active Learning Laboratory and Lecture Curricula. American Journal o f
Physics, 66(4), 338.
Transforming Undergraduate Education in Science, Mathematics, Engineering,
and Technology. (1999). Washington, D.C.: IVational Academy Press.
Tytler, R.
(2002).
Teaching for understanding in science: Student conceptions
research, and changing views of learning. Australian Science Teachers'
Journal, 48(3), 14-21.
van Roon, P. H., van Sprang, H. F., & Verdonk, A. H. (1994). 'Work' and 'Heat':
on a road towards thermodynamics. lnternational Journal o f Science
Education, 76(2), 131-144.
Von Secl<er, C. E., & Lissitz, R. W. (1999). Estimating the Impact of Instructional
Practices on Student Achievement in Science. Journal of Research in
Science Teaching, 36(10),
Wee, A. T.
1110-1126.
S., Baaquie, B. E., & Huan, A. C. H. (1993). Gender differences in
undergraduate physics examination performance and learning strategies
in Singapore. Physics Education, 28, 158-163.
Willden, J. L., Crowther, D. T., Gubanich, A. A., & Cannon, J. R.
(2002).
A
Quantitative Comparison of Instruction Format o f Undergraduate
Introductory Level Content Biology Courses: Traditional Lecture
Approach vs. Inquiry Based for Education Majors. Paper presented at the
Annual International Conference of the Association for the Education of
Teachers in Science, Charlotte, NC.
Yarroch, W. L. (1985). Student Understanding of Chemical Equation Balancing.
Journal of Research in Science Teaching, 22(5), 449-459.
APPENDICES
APPENDIX A
FREE RESPONSE SURVEY
I. What is energy?
Plants make their own energy. Do you agree or disagree w i t h this
statement?
2.
3. Please explain the reasoning you used t o answer Question
2.
4. In terms of energy, define "open system" and "closed system".
5. What is meant by the phrase "conservation of energy", i n terms of scientific
systems?
6. The first law of thermodynamics doesn't have t o apply t o biological systems.
Do you agree or disagree with this statement?
Explain your reasoning
APPENDIX B
PRE-TEST SURVEY
Please answer t h e following questions t o the best o f your ability. This
questionnaire will n o t be graded, nor will participation affect your course grade.
However, your participation is greatly appreciated.
I. This course is:
A: B10 100
B: GES 101
C: CHY 121
D: PHY 105
r. Please indicate i f you are taking a class this term in any other science, other
than this course.
A: none
B: biology
C: earth sciences
D: chemistry E: physics
3 . W h a t semester of your undergraduate studies are you i n ?
A: first
B: second
C: third
D: f o u r t h
E: f i f t h or
higher
4. Please indicate, i f Itnown, your intended major.
A: Biological Sciences
Physics
E: non-science
B: Earth Sciences
C: Chemistry D:
5. If this is n o t your first semester at the university level, please indicate any
fields o f science in which you have taken a course prior t o this term.
A: none
B: biology
C: earth sciences
D: chemistry E: physics
6. Energy can best be described as?
A: a force needed t o do work
B: heat
C: worlt done on an object
D: an ability t o do work
E: an interaction between molecules
7 . In closed systems energy is
A:
B:
C:
D:
E:
, and in open systems energy is
stored t o be used at a later time; cannot be stored f o r later use
stays in the system; doesn't stay in the system
limited; unlimited
conserved; isn't conserved
recycled; lost
8. W h a t is the mass o f the solution when I Icilogram o f salt is dissolved in 20
Itilograms o f water?
A: 19 Itilograms.
B: 20 Itilograms.
C: Between 20 and 21 Itilograms.
D: 21 ltilograms.
E: More than 21 Itilograms.
9. True or False? When a match burns, some mass is destroyed.
A: True
B: False
10. What
is the reason for your answer t o question 9 ?
A: This chemical reaction destroys mass.
B: Mass is consumed by the flame.
C: The mass of ash is less than the match i t came from.
D: The atoms are not destroyed, they are only rearranged.
E: The match weighs less after burning.
11. Which
of the following must be the same before and after a chemical
reaction ?
A: The sum of the masses of all substances involved.
B: The number of molecules of all substances involved.
C: The number of atoms of each type involved.
D: Both (a) and (c) must be the same.
E: Each of the answers (a), (b), and (c) must be the same.
Biological systems are often considered open systems, so energy
conservation does not apply.
B: False
A: True
12.
13. What is the reason for your answer t o question 12?
A: energy in open systems is lost.
B: energy in open systems is unlimited.
C: energy in open systems is exchanged with the outside.
D: energy in open systems is transferred t o other things.
E: energy in open systems is converted into matter.
14. Two billiard balls of equal masses are on a level, frictionless surface. The
first ball is moving and collides with the second ball, which was stationary. After
the collision, both balls are moving. What is the speed of the first ball after the
collision ?
A: less than its original speed.
B: the same as its original speed.
C: more than its original speed.
D: there isn't enough information t o determine an answer.
15. The 'Water Cycle' is an example of:
A: an open system.
B: a closed system.
16. What is the reason for your answer t o question 15?
A: conservation of water.
B: conservation of energy.
C: conservation of mass.
D: conservation of luck (I guessed).
E: conservation of natural resources.
17. The initial phase of an avalanche can be used as an example of conservation
of energy because i t represents:
A: change in ltinetic energy t o potential energy
B: change in thermal energy t o mechanical energy
C: change in potential energy t o kinetic energy
D: change in thermal energy t o gravitational energy
E: change in gravitational energy t o ltinetic energy
Do you agree or disagree with the following statements?
.
The law of conservation of energy applies t o the course I am taking.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E: strongly agree
19. The law of conservation of energy will be taught in detail in the course I am
taking.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E : strongly agree
20. The law of conservation of energy is relevant t o the field of science this
course is about, but isn't relevant t o this class specifically.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E : strongly agree
APPENDIX C
POST-TEST SURVEY
Please answer the following questions t o the best of your ability. This
questionnaire will not be graded, nor will participation affect your course grade.
However, your participation is greatly appreciated.
I. This course is:
A: B10 100
B: GES 101
C: CHY 121
D: PHY 105
2. Please indicate if you are taking a class this term in any other science, other
than this course.
A: none
B: biology
C: earth sciences
D: chemistry E: physics
3. What semester of your undergraduate studies are you i n ?
A: first
B: second
C: third
D: fourth
E: fifth or
higher
4. Please indicate, if Icnown, your intended major.
A: Biological Sciences
Physics
E: non-science
B: Earth Sciences
C: Chemistry D:
5. If this is not your first semester a t the university level, please indicate any
fields of science in which you have talcen a course prior t o this term.
C: earth sciences
D: clieniistry E: physics
A: none
B: biology
6. Energy can best be described as?
A: a force needed t o do work
B: heat
C: worlc done on an object
D: an ability t o do work
E: an interaction between molecules
7. In closed systems energy is
A:
B:
C:
D:
E:
, and in open systems energy is
stored t o be used a t a later time; cannot be stored for later use
stays in the system; doesn't stay in the system
limited; unlimited
conserved; isn't conserved
recycled; lost
8. What is the mass of the solution when
Icilograms of water?
A: 19 kilograms.
B: 20 Icilograms.
C: Between 20 and 21 Icilograms.
D: 21 Icilograms.
E: More than 21 Icilograms.
Ileilogram
of salt is dissolved in 20
9. True or False? When a match burns, some mass is destroyed.
A: True
B: False
10. What is the reason for your answer t o question 9?
A: This chemical reaction destroys mass.
B: Mass is consumed by the flame.
C: The mass of ash is less than the match i t came from.
D: -The atoms are not destroyed, they are only rearranged.
E: The match weighs less after burning.
11. Which of the following must be the same before and after a chemical
reaction ?
A: The sum of the masses of all substances involved.
B: -The nurnber of molecules of all substances involved.
C: The number of atoms of each type involved.
D: Both (a) and (c) must be the same.
E: Each of the answers (a), (b), and (c) must be the same.
12. Biological systems are often considered open systems, so energy
conservation does not apply.
A: True
B: False
13. What
A:
B:
C:
D:
is the reason for your answer t o question 12?
energy in open systems is lost.
energy in open systems is unlimited.
energy in open systems is exchanged with the outside.
energy in open systems is transferred t o other things.
E : energy in open systems is converted into matter.
14. Two billiard balls of equal masses are on a level, frictionless surface. The
first ball is moving and collides with the second ball, which was stationary. After
the collisio~i,both balls are moving. What is the speed of the first ball after the
collision?
A: less than its original speed.
B: the same as its original speed.
C: more than its original speed.
D: there isn't enough information t o determine an answer.
15. The'Water Cycle' is an example of:
A: an open system.
B: a closed system.
16. What is the reason for your answer t o question 1 5 ?
A: conservation of water.
B: conservation of energy.
C: conservation of mass.
D: conservation of luck (I guessed).
E: conservation of natural resources.
17. The initial phase of an avalanche can be used as an example of conservation
of energy because i t represents:
A: change in Itinetic energy t o potential energy
B: change in thermal energy t o mechanical energy
C: change in potential energy t o ltinetic energy
D: change in thermal energy t o gravitational energy
E: change in gravitational energy t o Ikinetic energy
Do you agree or disagree with the following statements?
18. The law of conservation of energy applied t o the course I am taking.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E: strongly agree
19. The law of conservation of energy was taught in detail in the course I am
taking.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E: strongly agree
20. The law of conservation of energy is relevant t o the field of science this
course is about, but was not relevant t o this class specifically.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E: strongly agree
APPENDIX D
ITEM ANALYSIS
Sub-Group
8
9
10
11
12
13
14
15
16
17
All Students:
% pre-test
65
65
66
35
64
18
77
51
34
43
All Students:
% post-test
66
72
72
28
69
26
78
53
45
79
AllStudents:<g>
0.03
0.20
0.18
0.14
0.10
0.04
0.04
0.17
0.63
Biology:
pre-test
63
60
61
34
64
18
78
45
30
34
Biology:
% post-test
60
69
69
24
67
26
76
51
45
73
Biology: <g>
-0.08
0.23
0.21
-0.15
0.08
0.10
-0.09
0.11
0.21
0.59
Chemistry:
% pre-test
83
72
72
33
89
11
89
72
28
94
Chemistry:
% post-test
89
89
89
50
61
22
78
67
22
89
Chemistry:<g>
0.35
0.61
0.61
0.25
-2.55
0.12
-1.00
-0.18
Earth Science:
% pre-test
75
70
73
45
70
30
77
68
60
79
Earth Science:
% post-test
55
57
57
30
64
34
75
59
64
75
-0.43
-0.59
-0.27
-0.20
0.06
-0.09
-0.28
0.10
-0.19
O/O
EarthScience:<g> -0.80
I
-0.08 -0.83
Physics:
% pre-test
64
64
57
21
64
29
79
57
50
SO
Physics:
% post-test
43
64
64
36
57
7
86
50
29
93
Physics: <g>
-0.58
0.00
0.16
0.19
-0.19
-0.31
0.33
-0.16
-0.42
0.86
Bio-Chem:
% pre-test
62
64
65
36
58
19
74
52
32
35
Bio-Chem:
% post-test
73
78
75
30
74
24
82
51
42
86
Bio-Chem:<g>
0.29
0.39
0.29
-0.09
0.38
0.06
0.31
-0.02
0.15
0.78
APPENDIX E
H U M A N SUBJECTS RESEARCH PROPOSAL
Measuring Student Learning of Conservation of Mass and Energy in
Introductory University Science Courses.
I. Summary of the Proposal
This is a proposal t o study how well students in their first semester of
university-level science courses learn the conservation of mass and energy. For
most university degrees, several semesters of science are required, but specific
subjects are not. Therefore, students can choose which course they wish t o take
first. This study would look into both how students learn the concept of
conservation in the 4 science courses (Biology, Chemistry, Earth Science, and
Physics) as well as investigate any pattern of choice of subject t o meet the basic
requirements for an undergraduate degree.
Students would be asked t o fill out a questionnaire during the first week
of their Fall-semester science course, and one again near the end of the
semester. The questionnaire would include questions about conservation of
mass/energy, subject-specific applications of conservation of mass/energy, and a
section on how well the students feel the conservation idea relates t o their
particular course. The post-test version would ask how well the topic was
covered in class. In addition t o the subject-related questions, students would be
asked why they chose that particular course.
The results would be interpreted on several levels. First, each course
would be analyzed for overall gain in understanding the scientific concept. As
well, gain would be compared between each course. The qualitative aspects of
how well material related t o conservation of masslenergy and how i t was
presented would be analyzed in a similar fashion; for each course independently
and courses would be compared. Finally, any preference for choosing particular
courses would be noted.
The questionnaire will be in a multiple-choice and scaled-response format.
The questions were be developed with the help of experts in each field t o be
investigated t o insure relevance t o the material covered in the introductory
course. The questions may be slightly altered t o better f i t courses curricula, and
post-test questions in the final section w ~ lreflect
l
how students felt the topic
was taught.
Personnel
Jessica L. Odell, Master's candidate, University of Maine
Dr. Stephen Norton, Department of Earth Sciences, University of Maine
Dr. Michael Wittmann, Department of Physics and Astronomy, University
of Maine
Dr. Francois Amar, Department of Chemistry, University of Maine
Dr. Mary Tyler, Department of Biological Sciences, University of Maine
2.
3. Subject Recruitment
Students in each of the four introductory science classes participating in
the study will be asked t o participate on a fully voluntary basis. A brief
introduction will be made t o each of the participating classes by the investigator
at the tinie the informed consent forms are handed out.
4. Informed Consent
Each student willing t o participate in the study will be asked t o sign a
consent form describing his or her specific role in the study. Any minors a t this
level will be excluded from the study.
5. Confidentiality
All documents will be stored in a loclted office on the University of Maine
campus, either that of Jessica Odell or Stephen Norton. All electronic data will
be only accessible t o the investigators. Any identifying information, such as
names and ID numbers, will be replaced by code-numbers upon data processing.
All hard copies of documents w i t h identifying information will be destroyed
within 6 months of the completion of the study. In the case of publication,
there will be no subject-identifying information used.
6. Rislts t o Subjects
There is very little risk involved in this study. Questionnaires will be in
the format of typical tests and quizzes found 1
i 1 most university level classes.
Students will only be required t o take the pre- and post-test one time each, if
they agree t o participate.
As the questionnaires will be answered during class-time, they will be as
short as possible, hopefully doable in about 15 minutes. N o time will be required
of students outside of class.
7. Benefits
The primary benefit of this study will be t o the instructors of
introductory-level science classes at the University of Maine, particularly those
teaching students fulfilling only the basic core science requirement. They will
receive information on how effective their courses are in teaching some of the
basic scientific concepts, and information on how students perceive the
relevance of the concepts t o the course.
The University may benefit as well, in evaluating the goal of the core
science requirement, and the possible differences in learning basic concepts
between different fields.
Sample Questions:
The questionnaire will include approximately 20 questions. 5 ask about
the course the student is in, and other science classes taken at the University
level. 10 aslt about conservation of mass/energy, and the remaining 5 will ask
about how students feel the concept applies t o the field and how well i t was
taught.
What semester of your undergraduate studies are you in?
A: first
B: second
C: third
D: fourth
E: f i f t h or higher
Energy can best be described as?
A: a force needed t o do work
B: heat
C: worlc done on an object
D: an ability t o d o w o r l <
E: an interaction between molecules
True or False? When a match burns, some matter is destroyed.
A: True
B: False
What is the reason for your answer t o question 9 ? (the previous question)
A: This chemical reaction destroys matter.
B: Matter is consumed by the flame.
C: The mass of ash is less than the match i t came from.
D: The atoms are not destroyed, they are only rearranged.
E: The match weighs less after burning.
The First Law of Thermodynamics applies t o the course I am taking.
A: strongly disagree
B: disagree
C: neither disagree nor agree
D: agree
E: strongly agree
Informed Consent Form
You are invited t o participate in a research project being conducted by
Jessica Odell, a graduate student in the Master of Science i n Teaching Program
at the University of Maine. -The purpose of the research is t o investigate
understanding of various concepts and t o see how effectively those concepts are
presented in introductory science courses. You must be at least 18 years of age
in order t o participate.
W h a t will you be asked t o do?
The data for this study will come from 2 short questionnaires given
during the semester, one at the beginning and one a t the end of the term. These
questionnaires will not be graded, nor affect your grade in the course in any
fashion. Because this first questionnaire is very important t o this study, we ask
that you take i t just as seriously as you would a regular test. Although your
name will be present on the questionnaire, no personal identification (i.e., your
name or University identification number) will be used i n connection w i t h the
data.
RISKS
Except for your time and the effort in completing the questionnaires,
there are no foreseeable rislcs t o you in participating in this study.
Benefits
The study will help show how well the four introductory science classes
approach the first law of thermodynamics, and how students feel it was
presented. The results will be beneficial t o the University and the instructors in
evaluating how the current curricula approach this concept.
Confidentiality
Although your name will be on the questionnaires, your name will not be
on any of the documents developed in this study. A code number will be used t o
protect your identity. Data will be kept in Jessica Odell's locked office, and only
Jessica Odell and Prof. Norton will have access t o t h a t data. The key linking
your name t o the coded data will be destroyed after data analysis is complete.
The anonymous coded data will be kept for future use.
Your participation in this study is voluntary. Signing this form gives
permission t o use your future responses as data in this study. I f you choose t o
let your responses be used in this study, you may change your mind a t any time
during the semester. Simply contact any of the researchers listed below and aslc
that your responses be omitted from the study. Choosing t o participate, or not
t o participate, in the study will have no bearing on your grade for the course.
M a y we use your responses from the study questionnaires, without your
name or an other identification attached, for education research that may
be publishe ?
UP9
J
Signature:
Date:
Contact information (please keep this page for your records)
If you have any questions or concerns about this study, please contact
Jessica Odell at:
(official form will list my office and campus extension, which are
currently unknown)
Ph: 207-581Jesica.Odell@mutmaine ~ d u
You may also reach the faculty advisor for this project, Prof. Stephen
Norton, at:
3-14Bryand Global Sciences Center, University o f Maine, Orono, M E
04469
Ph: 207-581-2156
Norton@mdne e d u
I f you have any questions concerning your rights as a participant, please
contact Gayle Anderson, Assistant t o the University of Maine's Protection
of Human Subjects Review Board, at:
Ph: 207-581-1498,
or
Gayle A n d e r s o ~ u m i t . m a i n e . e d u
BIOGRAPHY OF THE AUTHOR
Jessica L. Odell was born in Miami, Florida on October 23,1975. She
graduated as valedictorian from Dr. Phillips High School in Orlando, Florida in
1993. In 1997 she graduated from Eckerd College with a Bachelor of Science in
Biology, and minored in Chemistry. While at Ecl<erdCollege, she participated in
research on bottlenose dolphins under Dr. John E. Reynolds, I I I and presented
posters on her research at the Eleventh Annual Conference on the Biology of
Marine Mammals (Orlando, Florida 1995) and at the World Marine Mammal
Science Conference (Monaco 1998). Jessica worked teaching test preparation
courses for The Princeton Review before entering graduate school a t The
University of Maine. She is a candidate for the Master of Science in Teaching
degree from -The University of Maine in August,
2005.