Fallacies in academic writing
Chad Nilep
There are many possible sources of fallacy – an idea that is mistakenly thought to be true,
even though it may be untrue – in academic writing. The phrase logical fallacy often refers to a
formal fallacy in a logical argument; that is, an invalid argument that is mistakenly thought to be
valid. Often people talk about various kinds of rhetorical fallacies, writing or speech that appears
to be based on an argument but is not logically entailed by premises. Research writing can also
be prone to statistical fallacy: mistaken use of statistics, or conclusions based on misunderstanding statistics. Below I discuss a few common fallacies in academic writing.
Logical (formal) fallacies
Affirming the consequent. Implication is a valid form of logical argument. The truth of one
idea (call it an antecedent, meaning it comes first) may imply a consequence. If the antecedent is
true then the consequent (the thing that happens after or sometimes because of the antecedent)
must be true. For example, if a person is murdered that person is dead. This is necessarily true
because it is part of the meaning of murder. If Taro was murdered, we can conclude that he is
dead. That is a valid argument from implication.
But the opposite is not a valid argument: the truth of the consequent does not imply the truth
of the antecedent. If Taro is dead, we cannot conclude that he was murdered; he might have died
from a disease or an accident. The consequent does not imply the truth of the antecedent.
Denying the antecedent. On the other hand, if an antecedent implies a consequent and that
consequent is not true, then the antecedent must not be true. Think about the murder example
again: if Taro was murdered, he is dead. If he is not dead, then we know that he was not murdered. But again the opposite is not a valid argument: a false antecedent does not imply a false
consequent. Even if we know that Taro was not murdered, we don’t know whether he is dead. He
may be alive, but he may have died naturally or accidentally.
Syllogistic fallacies
We discussed syllogisms in class. In a syllogism, two premises add up to a conclusion. If the syllogism is valid and the premises are true, then the conclusion must be true. Be careful, though:
there are several fallacies that can make a syllogism invalid.
VALID SYLLOGISMS
All P is M.
All M is Q.
Therefore, all P is Q.
Some P is M.
All M is Q.
Therefore, some P is Q.
All P is M.
No M is Q. (All M is not-Q)
Therefore P is not Q.
Undistributed middle. All syllogisms must have a middle term that is “distributed” – true for all
– in at least one premise. If the middle term is not distributed, the syllogism is not valid.
NOT VALID SYLLOGISMS
All P is M.
All Q is M.
Therefore, all P is Q.
Some P is M.
Some M is Q.
Therefore, some P is Q.
All P is M.
Some M is not Q
Therefore P is not Q.
We’ve seen the first non-valid argument in class: “All rabbits run fast; Usain Bolt runs fast.
Therefore, Usain Bolt is a rabbit.” I also mentioned an example of the second pattern: “Some
students in this class are women; some women are over 60 years old. Therefore, some students in
this class are over 60 years old.” Here is an example of the third pattern: “All students in this
class are graduate students. Some graduate students do not study logic. Therefore, students in
this class do not study logic.” None of the conclusions are proved true since the syllogisms are invalid.
Four terms. A valid syllogism must have three terms, like the ones labeled P, M, and Q above.
If the conclusion or one of the premises introduces a fourth term, it is not valid. Consider this
argument: “Socrates is a human. All humans are mortal. Therefore, Socrates is Japanese.” The
wrongness of that syllogism is obvious. But sometimes it can be hard to spot this fallacy when
the same words express two different ideas. Consider the example below.
Warm beer is better than nothing.
Nothing is better than true love.
Therefore, warm beer is better than true love.
Although this looks like a valid syllogism of the kind we have seen before, the conclusion is not
true. What went wrong? Notice that “better than nothing” (無いよりまし) in the first premise
does not mean the same thing as “nothing is better” (最高) in the second premise. Those two
different meanings add up to a fallacy of four terms.
Affirmative conclusion from negative premises or negative conclusion from affirmative
premises. Two negative premises cannot entail an affirmative conclusion. Likewise, it is not
valid to add negative meaning to an affirmative syllogism.
NOT VALID SYLLOGISMS
No P is M.
No M is Q.
Therefore, all P is Q.
Some P is M.
All M is Q
Therefore some P is not Q.
As an example of the first pattern consider this invalid syllogism: “No rabbits are horses; no
horses are birds. Therefore, all rabbits are birds.” It is not valid to draw an affirmative conclusion
from negative premises. Likewise, negative conclusions cannot be drawn from premises that
contain no negative statement. Maybe you can think of examples that seem to be true; for
example: “Some people are doctors; all doctors are university graduates. Therefore, some people
are not university graduates”. But the truth of that conclusion is only accidental. It is not a valid
conclusion from the syllogism. Consider this example, which uses the same logic: “Some army
officers are generals; all generals are soldiers. Therefore, some army officers are not soldiers.”
That’s not valid and it’s not true. Remember: a valid syllogism allows us to deduce that the
conclusion is true, but truth (matching the world) is not the same as validity (entailment from
argument).
Rhetorical (informal) fallacies
Writers sometimes argue, meaning they try to convince the reader that what they say is right,
without a logical argument in the sense of “a set of premises that imply the truth of, or support
the likelihood of their conclusion”. Here are some examples of rhetorical fallacy.
Appeal to authority. Writers may imply, or readers may assume, that something is true because
a famous, smart, or respected person said it. There is, however, no necessary link between the
authority of a speaker and the truth of what s/he says. Similarly a bandwagon argument
suggests that if many people believe the same thing, that belief must be true. Again, there is no
necessary link between popularity or fame and truth.
Ad hominem. When two people or groups disagree, an ad hominem argument may appear:
arguing that the person saying something is bad, so what s/he says must not be true. Just as the
goodness of a speaker does not entail the truth of what s/he says, the badness of a speaker does
not entail falseness.
Straw man argument consists of misrepresenting either a conclusion or a logical argument in
order to counter it. A scholar may reject another scholar’s conclusion by showing either that an
argument was invalid or its premises are not true. If instead the scholar creates an invalid
argument that reaches the same conclusion in order to reject that conclusion, he has created a
straw man argument.
Begging the question or circular reasoning involve making the conclusion one of the premises.
Consider this argument: “Fundamentals of academic writing is a popular class. Most students
like popular classes. Therefore, most students like fundamentals of academic writing.” The
conclusion and the first premise mean basically the same thing, even though the words are
slightly different. Therefore, it is not actually an argument; it just states what the teacher hopes is
true. (Note that in everyday English the phrase beg the question is sometimes used to mean
“cause to ask a question”. That is not related to this fallacy.)
Appeal to ignorance refers to the assumption that something not proved to be true must be false,
or that something not proved to be false must be true. Neither of these is a valid assumption;
something not proved might be true or false, but it’s currently unknown. For example, there is no
clear evidence that life exists on other planets, but that is not proof that no such life exists. It
might not exist, or it might exist but be unknown to humans. (Note, however, that some kinds of
absence can be used to support some kinds of inferences. For example if a doctor tests me for
cancer cells and doesn’t find any then it is likely, though not certain, that I do not have cancer.)
And although it’s not exactly a fallacy I will also share a warning from my former professor
Lise Menn: “Beware Procrustes bearing Occam’s razor”. Occam’s razor is the valid
suggestion (not a fallacy) that when two theories can account for the same data, it is best to
assume the simpler of the two theories. But simple theories are not always correct. Procrustes is a
character from a Greek myth who made everyone fit into the same bed by either stretching them
or cutting off their legs. Scholars sometimes commit an error by “cutting off” some data in order
to make it fit with a simple theory. For example, there is a simple theory that explains the
evolution of large human brains, but this theory cannot explain variation in the size of other
human-like species’ brains. In this case a more complex theory that explains all the data might be
true, even though it fails the suggestion of Occam’s razor.
Statistical fallacies
Readers and writers sometimes get confused about the meaning of statistical analyses. This
can lead scholars to claim things that are not supported by the analysis. Here are some statistical
fallacies.
Correlation is not causation. Correlation means that two values vary together. Causation means
that a change in one variable causes a change in another. For example, an increase in demand for
sugar can cause the price of sugar to increase. A common error is to assume that because values
are correlated (for example, when one increases the other also increases) one must cause the
other. Sometimes correlation is spurious or accidental. For example, rates of cancer and rates of
autism have both increased over the past twenty years, but it is unlikely that these changes are
related. Sometimes a third factor – called a hidden variable or a confounding variable –
correlates with both variables. For example, drowning deaths are strongly correlated with ice
cream sales; this is not because ice cream causes drowning but because both ice cream and
swimming are most popular in summer. This is an example of a common informal logic fallacy
called cum hoc ergo propter hoc “with this therefore because of this”. Correlation does not
imply causation. On the other hand, it often suggests that a phenomenon might be worth investigating further to see if there is some hidden meaning behind the correlation.
Significance versus “significance”. In everyday usage the word significant means “important,
noteworthy, or meaningful”. But in statistics significant means “unlikely to be a random error”.
The value p<0.01 calculated by a t-test essentially means that if our numbers were completely
random, the difference we found would be seen less than one time in 100. People sometimes
misunderstand, thinking that a statistically significant result must be important or meaningful.
That is not necessarily the case. For example, say we give an intelligence test to 10 men and 10
women and find that the average IQ of the men is 105 while the average of the women is 107.
This difference is not significant (p=0.15). But say we give the same test to 100 men and 100
women and find the same averages: 105 for men, 107 for women. This result is statistically
significant (p<0.01), even though the result is no more or less meaningful. This is because a t-test
is sensitive to small differences when sample sizes are large. Statistical significance allows us to
reject a null hypothesis – it tells us that there actually is a difference – but it cannot indicate how
important or meaningful that difference is.
Base rate fallacy. When doing statistical tests, people often ignore the “base rate” or underlying
probability and focus only on the probability of the test, leading them to under- or over-estimate
their findings. For example, imagine that there are 200 werewolves in Nagoya. Luckily, I have a
machine that can identify werewolves. The machine is 99% accurate, meaning it will misidentify
a werewolf as human only 1% of the time, and misidentify a human as werewolf only 1% of the
time. I point the machine at my neighbor and it says she is a werewolf. How confident can I be
that she really is a werewolf? If I ignore the base rate, I may think that there is a 99% chance that
she is a werewolf. But in fact, since there are about two million humans and only two hundred
werewolves in Nagoya, the machine will wrongly identify about 20,000 humans as werewolves
while accurately identifying 198 werewolves. There is actually less than 1% chance that the
neighbor identified as a werewolf really is one.
Biased samples. It is usually impossible or at least impractical to test every person or thing
relevant to our research. Therefore, we usually select a sample and then generalize the results to
a larger population. Such generalization is only valid, though, if the sample is representative of
the population. A random sample, in which participants are randomly selected from the whole
population and subjects are randomly assigned to treatment or control groups when doing an
experiment, can help ensure representative, unbiased samples. Using convenience samples,
participants who are not randomly selected, can make the results less generalizable. For example,
since many psychology studies are done by university professors, they often recruit university
students as subjects. But since university students are not representative of the whole world
(among other differences, they tend to be younger and better educated than average) the results
cannot be generalized to all people. Similarly, medical research carried out in hospitals may
choose subjects from among hospital patients and therefore ignore the large portion of the
population who are not sick.
Another form of sample bias is called self-selection or non-response bias. For example, if a
sociologist sends a questionnaire to a random sample of people, some of those subjects might not
answer. The sample becomes biased if the people who respond and the people who do not
respond differ in some way. Similarly, if a family restaurant asks customers to rate their service
it is likely that only people who are very happy or very unhappy with the restaurant will answer,
so that the survey results will not represent the whole population of customers.
Extrapolation fallacies. Research in many fields examines change over time (or across space, or
over other variation). A common use of such results is to extrapolate – to predict that the rate of
change observed in the past will continue into the future. This can be useful, but it can lead to
wrong predictions. For example, during the 2012 United States presidential campaign three
opinion polls in September found that candidate Mitt Romney was becoming more popular each
week, while President Barrack Obama was becoming less popular. News media predicted that
this trend would continue and that Romney would win the election in November. In fact though,
the trend changed in October: Obama became more popular, and he eventually won the election.
“Garbage-in-garbage-out” is a phrase commonly used in computer science and information
technology; it can also apply to statistics. It means that the value of an analysis (what comes out)
is only as good as the value of the data that goes into it. Readers and writers often assume that
statistical analyses are reliable and valid without questioning the quality of the data that was
analyzed or the relevance of the analysis to the research question. If the data are not accurate or
appropriate, statistical methods will not make them any more reliable.
Finally, a word about charts. When statistical data are presented in chart form, even accurate
numbers can give a false impression. Consider these two charts. Both of them show the exact
same data: an increase from 35.0% to 39.5%. In the chart on the left that change looks very big,
while on the right it looks small. That apparent difference is due to the scale of the charts, from
32-40 on the right or from 0-100 on the left. Choosing one of these charts rather than the other
can give a very different impression of the same descriptive statistics. Be careful to present your
data accurately and fairly.