Two Wrongs Make a Right
Two Wrongs Make a Right:
Using Pseudoscience and Reasoning
Fallacies to Complement
Primary Literature
By Shawn Stover
Undergraduate science students
benefit greatly by learning to read
and interpret primary research
articles. However, once they obtain
a level of competence in analyzing
primary literature and develop a
better understanding of the nature of
science, they may become frustrated
by the lack of scientific literacy
and objectivity demonstrated by
the general public during debates
of “controversial” scientific
topics. A novel course has been
developed to provide students with
the knowledge and skills required
to interpret primary literature and
appreciate the nature of science, as
well as a better understanding of
the unscientific thinking they will
encounter in the real world.
A
t Davis & Elkins College,
biology majors must complete BIOL 335 (Current
Topics in Biology), a
one-credit course that involves the
analysis and discussion of primary
research articles. Analyzing primary literature allows students to
increase their quantitative literacy
(Kozeracki, Carey, Colicelli, &
Levis-Fitzgerald, 2006), enhance
their critical thinking skills (Houde,
2000; Smith, 2001), and improve
their understanding of the nature of
science (Wenk & Tronsky, 2011) as
they participate in the dissemination
of scientific information.
Previous studies have introduced
the reading of primary literature to
first-year students (Wenk & Tronsky,
2011), used peer instruction to facilitate the integration of primary literature into courses (Jacques-Fricke, Hubert, & Miller, 2009), and developed
undergraduate journal clubs to teach
a systematic method for interpreting
primary literature (Robertson, 2012).
BIOL 335 provides undergraduate
students with an opportunity to sharpen their data analysis skills, while
learning to discriminate between
legitimate scientific evidence and
public conjecture. After participating in the course, students should be
better prepared to evaluate scientific
issues presented by the media.
Evaluating information
The initial meetings of BIOL 335
include an introduction to the hier-
archy of scientific evidence (Figure
1). Students taking the class are usually in their junior year and likely
have had some exposure to journal articles. They are also likely to
be familiar with the unreliability of
testimonials and other forms of anecdotal evidence, but may be less familiar with the differences between
case studies, observational studies,
experimental studies, and systematic
reviews. Case studies generally follow a single subject and may involve
very unusual symptoms or circumstances. Observational studies follow a number of subjects, either a
cohort with a shared characteristic or
a cross-section representing a varied
population, and involve no randomized assignment to experimental and
control groups. Experimental studies
involve the manipulation of variables
and are likely to be both controlled
and randomized. Systematic reviews
generally represent a compilation of
the results of multiple controlled,
randomized experimental studies
addressing a common topic (Greenhalgh, 2000).
Eventually, students in the class become quite familiar with the objective
nature of the scientific process. They
come to realize that opinions matter
very little in science; it’s the evidence
that counts. At that point, they can
become extremely frustrated with
the lack of objectivity demonstrated
by the general public when topics
like evolution and global warming
are being debated. To address their
Vol. 45, No. 3, 2016
23
concerns, BIOL 335 also includes
lively discussions of common reasoning mistakes. Although students may
be unfamiliar with the terminology
associated with reasoning fallacies,
they quickly come to recognize the
weak arguments presented during
informal debates of “controversial”
scientific topics. Examples include,
but are not limited to, the following:
an ad hominem argument is a personal
attack on an individual and does not
specifically address the topic; an appeal to emotion is an argument that
attempts to arouse the emotions of
an audience to gain acceptance of its
conclusion; an argument by elimination, or false dilemma, is a tendency to
frame an issue as “either–or,” ignoring other likely possibilities (Carey,
2004; Kida, 2006). Finally, students
may be unaware of concepts like
identity bias and confirmation bias.
An individual’s sense of identity
can be greatly influenced by politics
and religion, and these “group allegiances” may cause an individual
to cling to certain beliefs, even in
the face of evidence that invalidates
them (Cohen, 2012; Stover, 2014).
Identity bias is a subcategory of
confirmation bias, a type of selective
thinking whereby an individual tends
to focus on information that confirms
his or her beliefs, while ignoring or
undervaluing the relevance of information that contradicts his or her
beliefs (Skeptic’s Dictionary, 2014).
Instead of forming conclusions based
on empirical evidence, individuals
exhibiting identity bias often just fall
in line with group ideology.
Analyzing articles
Topics covered in BIOL 335 tend to
be scientific issues considered controversial by the general public, as
well as supernatural or paranormal
issues like pareidolia (seeing facelike patterns in inanimate objects),
near-death experiences, and alien
abductions. An understanding of
reasoning fallacies seems to make it
24
Journal of College Science Teaching
FIGURE 1
Hierarchy of scientific evidence.
easier for students to grasp the public’s denial of the scientific evidence
supporting concepts like biological evolution, anthropogenic global
warming, and childhood vaccinations. It may also help them recognize the appeal that supernatural or
pseudoscientific beliefs hold for the
general public. Table 1 contains a
few topics that have been covered in
recent years. Each topic is represented by a specific type of article. We
generally start with a blog post regarding a scientific issue. Although
the blogger may be a respected scientist who carefully cites legitimate
sources, students are expected to
recognize that the blog post itself is
not an example of primary literature
and cannot be considered a reliable
source of information. We then analyze a published case study, usually
biomedical in nature. Students are
expected to appreciate the value
of carefully documenting a rare or
novel medical condition, but they
are also expected to realize that the
results of case studies may not extrapolate to a general population. We
read and analyze an observational
study. Students come to understand
that experimental studies may be impractical, or even unethical, at times,
facilitating the need for a cohort or
cross-sectional study with no control
or experimental groups. We spend
most of our time on experimental
studies, using a specific analysis
tool (Table 2) to dissect each article.
Students generally become quite adept at identifying hypotheses (even
when they are not explicitly stated),
interpreting data, and suggesting
follow-up studies. We finish up with
a review article. A single published
experiment in a specific scientific
discipline is really just the beginning
of the scientific process. That experiment will be replicated and revised
by like-minded scientists all over the
world. A systematic review brings
all the relevant data that has survived
peer review and all the relevant hypotheses that have been supported
by multiple studies, and combines
them into a single article. Students
quickly come to appreciate the value
of a systematic review, especially as
they consider their capstone projects
for the next year.
Recognizing pseudoscience
To introduce each topic, the class
views an episode of Penn & Teller’s
Showtime TV series Bullshit! that
debunks pseudoscientific claims and
paranormal phenomena. The show
ran for eight seasons (from 2003
Two Wrongs Make a Right
to 2010) and is available on DVD
and YouTube. Because of excessive profanity and occasional nudity,
the Penn & Teller series is intended
for mature audiences. Students are
asked to read and sign a participant
consent form (approved by the college’s Institutional Review Board) if
they wish to view the Penn & Teller
episodes. Students who are uncomfortable with the strong language
can opt out of viewing the episodes,
while still participating in the analysis and discussion of journal articles.
In the 5 years we’ve used the videos,
no one has ever opted out. In fact, the
Penn & Teller episodes are generally
considered one of the highlights of
the class, as indicated by representative comments on course evaluation forms (Table 3). Penn & Teller
are magicians and entertainers, not
scientists. They have very strong
opinions, and they usually reinforce
those opinions with copious amounts
of scientific evidence. Episodes dealing with near-death experiences,
alien abductions, genetically modified foods, “alternative” medicine,
pareidolia, and childhood vaccinations are exceptional. The prevailing
scientific evidence, or lack thereof, is
undeniable in those episodes. However, Penn & Teller occasionally let
their opinions get ahead of the science and that can be a good thing for
a class like BIOL 335. If students
are paying attention and are able to
integrate the knowledge and skills
they’ve obtained from previous
meetings of the class, they are able to
pick up on errors made by the hosts.
For example, the episode on Being
Green turns into an ad hominem attack on Al Gore, rather than a review
of the actual evidence supporting anthropogenic global warming. There
are also problems with episodes
TABLE 1
Representative topics for each type of article.
Topic
Type of article
Title
Obesity trends in the
United States
Expert opinion
What’s causing the obesity epidemic? (Novella, 2011)
The neuroscience of neardeath experiences
Case study
Stimulating illusory own-body perceptions (Blanke, Ortigue, Landis, & Seeck,
2002)
Alien abductions and
sleep paralysis
Observational
study
Sleep paralysis in adolescents . . . (Jiménez-Genchi, Avila-Rodriguez, SánchezRojas, Terrez, & Nenclares-Portocarrero, 2009)
The safety of genetically
modified foods
Experimental
study
Allergenicity assessment of Allium sativum leaf agglutinin . . . (Mondal et al.,
2011)
Evidence supporting
biological evolution
Systematic
review
Natural selection drives the evolution of ant life cycles (Wilson & Nowack, 2014)
TABLE 2
Instructions for analyzing experimental studies (based on an instrument created by Dr. Catherine Gardiner,
University of Northern Colorado).
Question
Description
Citation?
Author’s last names and first initials, year of publication, title of article, name of journal, volume number, issue
number, first/last page numbers.
Gap in
knowledge?
What biological question does the research address? In other words, what was known and unknown prior to
this research? This information is generally found in the Introduction.
Overall
hypothesis?
A statement of explanation regarding the research question. A hypothesis may not be clearly stated. It may
have to be inferred based on the procedures used to address the research question.
Prediction?
An “If . . . , then . . .” statement. If the hypothesis is supported, what results are expected?
Methods?
What was measured or determined? Summarize the approach in your own words.
Results?
What new information was produced? Summarize the results in your own words.
Conclusion?
What do the authors make of the data? Are their conclusions valid? Do the data support the hypothesis? Is
there any other possible interpretation?
Now what?
A good paper may generate more questions than it answers. What is the next question researchers in the field
should address?
Vol. 45, No. 3, 2016
25
TABLE 3
Representative comments from participating students.
Term
Comment
Spring 2010
“I especially enjoyed the Penn & Teller videos. They contributed to the overall learning of the subject and
entertained me. They made me enjoy coming to class every week because of their relevance.”
Spring 2010
“This class brings together concepts from my studies and teaches me to apply them to the real world. The
shows really point out how to think critically about science.”
Spring 2012
“Very interesting topics discussed this term. Dr. Stover encouraged us to think critically and to use our
knowledge of logic and science to our advantage.”
Spring 2012
“Really good class; it was interesting and opened my eyes to how some people view certain scientific topics.”
Spring 2014
“This class was interesting and it gave me a better understanding of the knowledge of the general public.”
Spring 2014
“The way in which this current topics course was taught was very thorough. It bridged the gap between
pseudoscience and science in a way that provoked thoughtful comments. It also encouraged students to use
their brains, whether it was analyzing articles or discussing Penn & Teller.”
dealing with the dangers of secondhand smoke and the impact of genetics on weight loss. It’s good to have
examples of opinions supported by
science, as well as those that are not.
Students need to understand that a
loud, passionate argument is not necessarily a good argument.
Student reaction
In general, the course has been successful, as well as popular. Students
quickly learn to recognize the type
of study presented in a scientific
journal and are able to analyze the
data accordingly. They also become
much more cognizant of reasoning
errors and the effects that those errors have on the acceptance of scientific evidence. Comments from
course evaluations are generally
positive, most along the lines of
those presented in Table 3. Students
often mention that a lot of work is
required for this one-credit course.
I can’t argue with that. Occasionally, a student will complain that
there aren’t enough articles in his
or her specific area of interest (cell
biology, ecology, etc.), but most appreciate that the point of the class
is to give them the tools to interpret
any scientific paper, as well as some
insight into the rejection of evidence by the general public. Once
26
Journal of College Science Teaching
or twice, students have commented
that some of the topics we covered
weren’t quite current (a case study
from 2002, for example). I always
put an older article in the rotation
just to demonstrate that concepts
considered controversial by the
public (like evolution by natural
selection) may have decades of evidence supporting them. Conversely,
certain pseudoscientific topics may
have been popular for generations
(like astrology), even though no
empirical evidence exists in the
literature to support their validity.
Finally, I did receive one negative
comment regarding the Penn &
Teller videos. The student “didn’t
really like the Penn & Teller videos
because they always took one side
and downplayed the opposing view.
. . . They pushed one side of the
argument without giving the other
side equal time.” I took this comment very seriously, and I address
it each time I teach the course. It is
imperative for students to realize
that all opinions are not necessarily
equal. Equal time is not required if
all the evidence is on one side of the
argument. If one side demonstrates
a preponderance of scientific evidence, and the other side offers only
anecdote, it would be unfair to give
them equal consideration.
Conclusion
The primary objectives of the course
are to give students: (a) the knowledge and skills required to interpret
primary scientific literature; (b) an
understanding of how science actually works; and (c) the realization
that science is vastly different from
other, less objective ways of thinking. In addition, the course should
give students some insight into the
unscientific thinking that is so common in the world today.
In the future, we plan to collect
pre- and postcourse information from
participating students to determine
whether this particular approach can
improve critical thinking skills or
understanding of the nature of science. We will also collect information
regarding students’ perceptions of the
effectiveness of the course. Davis &
Elkins College is a very small, private
school. Classes are small, so it may
take a considerable amount of time
to generate enough data for statistical
significance. Ultimately, our goal is to
develop a critical thinking course that
will help prepare our undergraduates
for the next phase of their careers,
and beyond. ■
Acknowledgment
I would like to thank Dr. Michelle Mabry
at Davis & Elkins College for her signfi-
Two Wrongs Make a Right
cant contribution to the editing of this
manuscript.
References
Blanke O., Ortigue S., Landis T., &
Seeck M. (2002). Stimulation illusory own-body perceptions. Nature
419(6904), 269–270.
Carey, S. (2004). A beginner’s guide
to scientific method. Belmont, CA:
Wadsworth.
Cohen, G. (2012). Identity, belief, and
bias. In J. Hanson (Ed.), Ideology,
psychology, and the law. Oxford,
England: Oxford University Press.
Greenhalgh, T. (2000). How to read a
paper: The basics of evidence-based
medicine. London, England: BMJ
Books.
Houde, A. (2000). Student symposia on
primary research articles. Journal
of College Science Teaching, 30,
184–187.
Jacques-Fricke, B. T., Hubert, A., &
Miller, S. (2009). A versatile module
to improve understanding of scientific
literature through peer instruction.
Journal of College Science Teaching,
39(2), 24–32.
Jiménez-Genchi, A., Avila-Rodríguez, V.
M., Sánchez-Rojas, F., Terrez, B. E.,
& Nenclares-Portocarrero, A. (2009).
Sleep paralysis in adolescents: The
“a dead body climbed on top of me”
phenomenon in Mexico. Psychiatry and Clinical Neurosciences 63,
546–549.
Kida, T. (2006). Don’t believe everything
you think. Amherst, NY: Prometheus.
Kozeracki, C. A., Carey, M. F., Colicelli,
J., & Levis-Fitzgerald, M. (2006). An
intensive primary-literature–based
teaching program directly benefits
undergraduate science majors and
facilitates their transition to doctoral
programs. CBE—Life Sciences Education, 5, 340–347.
Mondal, H. A., Chakraborti, D., Majumder, P., Roy, P., Roy, A., Bhattacharya,
S. G., & Das, S. (2011). Allergenicity
assessment of Allium sativum leaf agglutinin, a potential candidate protein
for developing sap sucking insect
resistant food crops. PLoS One 6(11),
e27716.
Novella, S. (2011). What’s causing the
obesity epidemic? Retrieved from
http://www.skepticblog.org
Robertson, K. (2012). A journal club
workshop that teaches undergradu-
ates a systematic method for reading,
interpreting, and presenting primary
literature. Journal of College Science
Teaching 41(6), 25–31.
Skeptic’s Dictionary. (2014). Confirmation bias. Retrieved from http://www.
skepdic.com
Smith, G. R. (2001). Guided literature
explorations: Introducing students
to the primary literature. Journal
of College Science Teaching 30(7),
465–469.
Stover, S. K. (2014). Changing long held
beliefs. Skeptic, 19(3), 54–56.
Wenk, L., & Tronsky, L. (2011). Firstyear students benefit from reading
primary research articles. Journal
of College Science Teaching, 40(4),
60–67.
Wilson, E. O., & Nowack, M. A. (2014).
Natural selection drives the evolution
of ant life cycles. Proceedings of the
National Academy of Sciences, USA,
111(35), 12585–12590.
Shawn Stover ([email protected]) is a
professor in the Department of Biology
and Environmental Science at Davis & Elkins College in Elkins, West Virginia.
Column Editors
If you are interested in submitting a manuscript to one of JCST’s columns or have a question or comment
regarding an article, please contact the appropriate editor at the following addresses:
The Two-Year
Community
Apryl Nenortas
[email protected]
Please submit directly
to JCST’s electronic
submission system
(http://mc.manuscriptcentral.com/nsta)
The Case Study
Clyde F. Herreid
Department of Biology
State University of New York
Buffalo, NY 14260-1300
[email protected]
Research and
Teaching
Ann Cutler
Editor
[email protected]
Please submit directly
to JCST’s electronic
submission system
(http://mc.manuscriptcentral.com/nsta)
Vol. 45, No. 3, 2016
27