COM 452 - Fall 2007
Building Communication Theory
Marc Le Pape
COM 452 - September 25th
Introduction to Validity:
Why we need to do certain things in order
to assure quality in theory building.
Validity
 The best available approximation to
the truth of a given proposition,
inference, or conclusion.
How does research
inform theory?
 Construct Validity
 Internal Validity
 External Validity
best
Construct
Validity
 To
what extend are the constructs of
theoretical
interest
successfully
operationalized in the research, i.e.,
 How do we measure what we want to measure?
 Each measure is called a variable
Construct
Validity
 The objective in ensuring construct
validity is to be able to measure the
constructs of interests in the hypothesis:
The causal construct, i.e., the cause.
The affected construct i.e., the effect
Construct
 The
Validity
variable operationalizing
causal construct is called:
The Independent Variable
the
Construct
 The
Validity
variable operationalizing
affected construct is called:
The Dependent Variable
the
Construct
Validity
 The affected variable is presumed to
be dependent on the causal variable
Construct
 In
theoretical research,
validity refers to:
Validity
construct
 The degree to which the independent
variable and the dependent variable
accurately measure the constructs of
interests in the hypothesis
Construct
Validity
 We say that a research design
has:
High construct validity
Low construct validity
Internal
Validity
 To what extent does the research
design allows us to reach causal
conclusions about the effect of the
independent
variable
on
the
dependent variable?
Internal
Validity
 The objective in ensuring internal
validity is to be able to confidently
argue that the associations stated in
the hypotheses are causal
Internal
Validity
 The internal validity of the research
refers to the degree to which
conclusions can be drawn about the
causal effect of the independent
variable on the dependent variable
Internal
Validity
 We say that a research design
has:
High internal validity
Low internal validity
External
Validity
 To what extent can we generalize
from the research sample and setting
to the population and setting specified
in the research hypothesis?
External
Validity
 The objective in ensuring external
validity is to be able to generalize the
results of the research to the
populations and settings of interests
in the hypothesis.
External
Validity
 The external validity of the research
refers to the degree to which the
results of the research can be
generalized.
External
Validity
 We say that a research design
has:
High construct validity
Low construct validity
Optimizing Construct Validity
 For empirical research to inform the
hypothesis, it must measure the
constructs to which the hypothesis
refer.
Optimizing Construct Validity
 If the observed variables do not have
construct validity, the research cannot
inform the theory.
Optimizing Construct Validity
 Typically a variable measures:
The construct of interest
+
Some constructs of disinterests
Optimizing Construct Validity
 Typically a variable’s measurement
also includes:
Random errors
Optimizing Construct Validity
 A variable with high construct validity
mostly measures the construct of interest,
with minimal contributions from:
 Constructs of disinterest
 Random errors
Optimizing Construct Validity
 To ensure high construct validity we
then need to measure a construct in
more than one way.
Optimizing Construct Validity
 Construct validity is best examined and
verified by:
 Using multiple operational definitions of the
construct
 Comparing
measurements to verify each
operational definition measures the same
thing
Optimizing Construct Validity
 If all measurements have similar results
we can be confident that all operational
definitions (i.e., all variables) measure,
among other things, the construct they
have in common – i.e., is the construct of
interest.
Optimizing
Internal
Validity
 Maximizes our ability to argue for
causal connections between the
independent variable and the
dependent variable
Optimizing
Internal
Validity
 A simple empirical association, or
correlation between independent and
dependent variables is not sufficient to
infer causality.
 Correlation does not imply causality!
Optimizing
Internal
Validity
 Inappropriately inferring causality
from a simple association is called:
 the correlational fallacy
Optimizing
Internal
Validity
 Assuming an association is causal there is
still uncertainty as to:
 Which construct is the cause
 which construct is the effect.
Optimizing
Internal
Validity
 If two constructs are causally related,
identifying the correct causal direction, is
crucial.
 Always watch for inappropriately drawn causal
inferences!
Optimizing

Internal
Validity
There could be 4 possible explanations
as to why two variables are associated:
1)
2)
3)
4)
XY
XY
X   Y (Reciprocal Causation)
Z X & ZY (Hidden Variable
Problem)
Optimizing
Internal
Validity
 To infer causality from a simple
association between variables is only
possible when:
 Participants to a study have been randomly
assigned to the independent variable
 or to the levels of the independent variable
Optimizing
Internal
Validity
 A research studies carried out in this
manner follows a randomized
experimental design and is called a :
 Randomized Experiment
 Such a design implies great control in assigning
participant to the levels of the independent variable
Optimizing
Internal
Validity
 A research studies that is not carried out in
this manner follows a quasi-experimental
design and is called a:
 Quasi-Experiment
 Such a design implies less control in assigning
participant to the levels of the independent variable.
Optimizing
Internal
Validity
 A quasi-experimental design does not
allow
causal inferences to be made with the
same degree of confidence as a
randomized experimental design does.
Optimizing
Internal
Validity
 Yet, although some internal validity is
sacrificed, a quasi-experimental design:
 Still can yield useful information
 often is the only alternative in social science
research
Optimizing External Validity
 Before the research is conducted it is
necessary to specify:
 the limits of desired generalization
Optimizing External Validity
 The more precise a theory is about:
settings and
population
 the easier the generalization
Optimizing External Validity
 A theory should never remain implicit
as to population and settings in its
hypotheses
Optimizing External Validity
 A theory should always be explicit
about:
The settings and the population for
which generalization is sought
The settings and the population for
which the theory hypotheses are
supposed to hold
Optimizing External Validity
 To enhance generalization,
theoretical research should always
draw:
 Representative Samples
Optimizing External Validity
 To be able to generalize with a high
degree of confidence from a sample to
a population of interest theoretical
research should always draw:
Random Samples.
Optimizing External Validity
 Random sampling
is not the same as
 random assignment:
Optimizing External Validity
 Random Sampling is done to enhance:
External Validity
 Random Assignment is done to
enhance:
Internal Validity
Optimizing External Validity
 Replicating research
in other settings
with different samples
 is important to enhance external validity
Optimizing External Validity
 Replicating research is important to
enhance external validity because:
 It is often impossible to draw a random
sample.
 If the result of the replication are consistent
with the original research a theory gains
increasing support for its hypotheses.
Validity
 Remember that the concept of validity
in theory formulation is a unifying
concept that explains why:
researchers need to do certain things in
order to assure quality in theory building.
Next Lecture: Measurements
 Definitional Operationism
 Theory Constructs & Measurements
 Components of an Observed Score
 Measurements Reliability
 Measurements Validity