Crawford School of Economics and Government
Causality and Causal
Inference
Semester 1, 2009
POGO8096/8196: Research Methods
28 April 2009 @ Crawford School
1
This week

Research design and causal thinking
What is a research design?
 Conditions of causality


Research designs for causal thinking
True experimental design
 Quasi-experimental design
 Correlation design
 (Design without a control group)

28 April 2009 @ Crawford School
2
What is a research design?

Narrow definition [today’s topic]


Research design refers to the logical method by
which we propose to test a hypothesis.
Broad definition

Research design refers to a whole proposal for a
research project, including the review of the
literature, research questions and hypotheses, details
of data collection, methods of data analysis,
expected outcomes, a budget proposal, etc.
28 April 2009 @ Crawford School
3
Why do we need it?



In empirical research, we often want to test a
causal theory (or more specifically, a causal
hypothesis) stating that X affects Y.
But … How do we know that X causes Y, but
not the other way around? How do we know
that it is X that causes Y, but not something else
(W, Z or whatever)?
We need a valid research design: a method with
which we can test the causal hypothesis.
28 April 2009 @ Crawford School
4
Conditions for causality



Co-variation (or Correlation): Two phenomena (i.e.,
variables) tend to be related. If one of the two variables
changes, another variable changes. [Necessary condition
for causality.]
Time-order: The presumed cause (an independent
variable) happened before the presumed consequence
(a dependent variable).
Non-spuriousness: The co-variation between independent
and dependent variables is not caused by other factors.
28 April 2009 @ Crawford School
5
Research designs for
causal inference

True experimental design


Quasi experimental design



Almost always quantitative
Often quantitative
“natural experiment” in Shively
Correlational design


Both qualitative (intentional selection of observations) and
quantitative (correlation and regression analysis)
“natural experiment without pre-measurement” in Shively
28 April 2009 @ Crawford School
6
True experiment – 1


Physical, biological and medical sciences, as well
as some social sciences (e.g., psychology), have
traditionally used true experiments. It is
becoming popular in other social science
disciplines.
It is very important to understand how an
experiment is set up, because the logic involved
is highly relevant to all types of research design
for causal thinking.
28 April 2009 @ Crawford School
7
True experiment – 2

The classic (simplest) experimental design
1.
2.
3.
4.
Select a sample of your experimental study.
Randomly divide the subjects (e.g., students) into
two groups: the “experimental (or treatment)
group” and the “control group.”
Give a “stimulus (or treatment)” only to the
experimental group.
Measure the dependent variable and compare the
difference in the dependent variable between the
groups.
28 April 2009 @ Crawford School
8
Note: In this
course,
“matching”
means
something
different (= the
intentional
selection of
observations).
“Random
Assignment”
28 April 2009 @ Crawford School
9
Types of true experiments

Laboratory experiment
Small-scale experiment in artificial setting
 The effects of negative TV campaigns on voter’s
preference of presidential candidates and on voter
turnout (Ansolabehere and Iyenger 1995).


Field experiment
Large-scale experiment in “real” setting
 The effects of personal canvassing, telephone calls,
and direct mail on voter turnout (Gerber and Green
2000).

28 April 2009 @ Crawford School
10
Merits



Well-structured experiments can control other factors
almost perfectly, because the attributes of experimental
and control groups (observable and unobservable) are,
on average, the same.
They also satisfy the “time-order” condition of
causality.
Thus, “only randomization [i.e., a true experiment with
random assignment] provides a clear enough causal
interpretation to settle issues of social-scientific
research conclusively” (Shively, p. 87).
28 April 2009 @ Crawford School
11
Limitations

In both laboratory and field experiments
Experimental research is often limited to
investigations of political and social communications.
 It may be ethically inappropriate to conduct
experiments with human beings, societies and
politics as subjects.


In laboratory experiments
Samples in an experiment may not represent the
population.
 Experiments are conducted in an artificial setting.

28 April 2009 @ Crawford School
12
Quasi experiment

The basic quasi experimental design
1.
2.
3.
Measure the dependent variable of the subjects.
Wait until some subjects are exposed to the
independent variable, or observe that the values of
the independent variable change among the
subjects. (Note: The subjects are not assigned to
groups: no random assignment.)
Measure the dependent variable again, and
compare the difference in the dependent variable
among the subjects.
28 April 2009 @ Crawford School
13
28 April 2009 @ Crawford School
14
Other examples


“Panel” studies – the same set of people are
surveyed multiple times.
“Before-and-after” studies in policy-oriented
research.
e.g., the effects of increasing speed limits on the
number of traffic fatalities. (Not all states in the US
increased the speed limits.)
 Note: There should be a “control group” in these
research.

28 April 2009 @ Crawford School
15
Merits and Limitations



They satisfy the “time-order” condition – The
value of a dependent variable changes after the
value of a key independent variable is observed
(or changed).
Important: They can control “subject-specific, timeinvariant” variables, but not others.
You should carefully examine what other factors
might affect the causal relationship and try to
control them (if possible).
28 April 2009 @ Crawford School
16
Correlational design

The basic correlational design
1.
2.
3.
Select a sample of your study.
Measure the dependent and independent variables.
(Note: The values of the independent variables
must vary among the subjects.)
If the dependent variable differs among the
subjects, ascribe this to the effect of the
independent variable.
28 April 2009 @ Crawford School
17
28 April 2009 @ Crawford School
18
Other examples



Studies examining how survey respondents with
different attributes vote differently.
Studies examining how countries with different
economic conditions experience different
environmental issues and conflicts.
Studies examining how Indonesian local
governments with different social conditions
affect their budget performance.
28 April 2009 @ Crawford School
19
Merits and Limitations



The “time-order” condition is not always
satisfied. We need “auxiliary” information (e.g.,
common sense, prior knowledge, etc.) to judge
the direction of causality.
A correlational design with just one independent
variable fails to control other factors.
You should carefully examine what other factors
might affect the causal relationship and try to
control them.
28 April 2009 @ Crawford School
20
Design without a control group

A typical design without a control group
1.
2.
3.
Measure a certain phenomenon or behavior (A)
you want to explain.
Observe a certain phenomenon, behavior or other
exogenous shocks (B) that you think as a “cause”.
Measure the phenomenon or behavior (A’) you
want to explain again. If the phenomenon or
behavior has changed from A to A’, then ascribe
this change to the occurrence of B.
28 April 2009 @ Crawford School
21
An Example

Taiwan presidential election in 2000.
The pre-election level of support for each candidate.
 China’s military actions near Taiwan, which are
intended to affect the election result.
 The results of the presidential election

The Democratic Progressive Party’s Chen Shui-bian won
 The ruling Kuomingtang’s Lien Chan lost.


Many argue that China’s attempts were
counterproductive and helped Chen to win. Really?
28 April 2009 @ Crawford School
22
Control group

Important: A control “group” and a control
“variable” are different. If there is no control
group, your independent variable does not vary.
A control group = a portion of your observations
(the subjects of your study) that are not exposed to
your key independent variable.
 A control variable = another factor explaining the
dependent variable.

28 April 2009 @ Crawford School
23
Examples

An example of true experiments:


An example of natural experiments:


Taking an introductory Australian Government course (X)
increases political interest (Y).  “Students who do not take
the course.”
Watching a presidential debate (X) increases intensity of
support (Y).  “Students who do not watch the debate.”
An example of correlational designs:

Voter turnout (Y) is lower in urban areas (X).  “Rural
areas.”
28 April 2009 @ Crawford School
24
Limitations


Research design without a control group is very
common in historical descriptions of political, social
and economic events. This method may be useful to
understand historical processes and details, but it is not
recommended for causal analysis. Why? Because
without a control group you cannot say that a variation
in Y is caused by a variation in X (because there is no
variation in X).
They may, however, help us identify some new
measures, theories, and/or puzzles.
28 April 2009 @ Crawford School
25
Summary
Random
Assignment
Premeasurement
Control
Group
True experimental
design
Yes
Yes/No
Yes
Quasi experimental
design
No
Yes
Yes
Correlational design
No
No
Yes
Design without a
control group
No
Yes
No
28 April 2009 @ Crawford School
26
Next week

Intentional selection of observations
How to choose observations?
 How to avoid problematic causal inference?


Some controversies
Observations vs. cases
 Objectives of qualitative research

28 April 2009 @ Crawford School
27