Quantitative Research Methods
Sofia Ramström
Medicinska vetenskaper, Örebro Universitet
Disposition
What is science – and what is
quantitative science?
II. How do we measure?
III. How do we determine the correctness
of our measurement?
I.
Diagnostikcentrum, klinisk kemi, Region Östergötland
What is good science and research?
Science – what it is
“Science is a way of thinking much more than it is a
body of knowledge” (Carl Sagan)
 ”A way of obtaining knowledge by means of
objective observations” (McBurney & White)
 A body of techniques for investigating phenomena
and acquiring new knowledge, as well as for
correcting and integrating previous knowledge.
 It is based on observable, empirical, measurable
evidence, and subject to laws of reasoning.

Basic assumptions of science
Inductive and deductive logic

A true, physical universe exists
Although there may be randomness and thus
unpredictability in the universe, it is primarily
an orderly system
 The principles of this universe can be
discovered, particularly through scientific
research
 Knowledge of the universe is always
incomplete. New knowledge can, and should,
alter current ideas and theories. Therefore, all
knowledge and theories are tentative


("top-down" approach)

Deduction
Induction
("bottom up" approach)
1
The goals of science

Categories of research
The discovery of regularities
 Quantitative
◦ Description
◦ Discovering laws (a statement that certain events are
regularly associated with each other in an orderly
fashion)
◦ Search for causes

 Qualitative research: do not attempt to quantify
their results through statistical summary or
analysis (Interviews and observations, case
studies.)
Development of Theories
◦ Theory (a statement or set of statements explaining
one or more laws, usually including one indirect
concept required to explain the relationship)
◦ Theories must be falsifiable
Quantitative science

Propose specific hypotheses as explanations of
natural phenomena

Design experimental studies that test these
predictions for accuracy.

vs. Qualitative
 Quantitative research: studies that make use of
statistical analyses to obtain their findings.
Quantitative Methods

Process of inquiry must be objective.

The researcher must make complete
documentation of data and methodology available for
careful scrutiny by other scientists and researchers.
These steps are repeated in order to make increasingly
dependable predictions of future results.

This also allows statistical measures of the
reliability of the results to be established.

Build theories that bind more specific hypotheses
together into logically coherent wholes.


This in turn aids in the formation of new hypotheses.
Attempt to achieve control over the factors
involved in the area of inquiry, which may in turn be
manipulated to test new hypotheses in order to gain
further knowledge.
Cornerstones of scientific methods
The Scientific Method

Empirical evidence (empiricism)
Logical reasoning (rationalism)
 Sceptical attitude (scepticism) about
presumed knowledge, being undogmatic
(willing to change one's beliefs)



Observation
Description
 Prediction
 Control
 Falsifiability
 Causal explanation
2
Falsifiability- elimination of plausible
alternatives
Hypotheses

An educated—and testable—guess about
the answer to your research question

Each hypothesis must make a prediction

These predictions are then tested, and
the hypotheses can either be supported
or refuted on the basis of the data
The hypothetico-deductive method


Demands falsifiable hypotheses (null
hypothesis), framed in such a manner that
the scientific community can prove them
false with a certain agreed probability
A gradual process that requires
repeated experiments by
multiple researchers who must
be able to replicate results
 All hypotheses and theories are in
principle subject to disproof
 There is a point at which there
might be a consensus about a
particular hypothesis or theory, yet
it must in principle remain
tentative.

Causal explanation

Requirements generally regarded as important
to scientific understanding:
◦ Identification of the causes of a particular
phenomenon.
◦ Covariation of events. The hypothesized causes
must correlate with observed effects.
◦ Time-order relationship. The hypothesized causes
must precede the observed effects in time.
If the null hypothesis is refuted by a certain
probability, the hypothesis is not necessarily
proven, but remains provisional
Serendipity
II. How do we measure?
Discoveries that are unanticipated, fortuitous, or
”lucky”
 Appear to be stumbled on while the scientist is
looking for something else.
 Once discovered they stimulate new theories
and/or research
 Seredipitous findings are not”happy accidents”.
They could easily have been missed had the
scientist not been alert to the implication of the
observation. Such alertness requires both a
prepared mind and a real sense of quriosity.

What do you measure in your research?

What is your current hypothesis?

How do you need to design your study to
be able to prove your hypothesis?

3
Measurement

Measurement is the assignment of
numbers to objects or events in a
systematic fashion.
”a set of operations having the object of
determining a value of a measurable quantity”
Variables

Aspect of a testing condition that can
change or take on different characteristics
with different conditions
Independent variable

The independent variables are those
that are deliberately manipulated to
invoke a change in the dependent
variables
Measurement

The set of values that can result from the
appropriate application of a particular
measurement/analytical procedure is
called possible values.

The measured quantity must be
expressed with both a value and a unit
Dependent variable

The dependent variables are those that
are observed to change in response to
the independent variables
Controlled variable
Condition or variable intentionally kept
constant throughout the study
 The purpose is to minimize the effects of
the controlled variable on the dependent
variables

4
Population
Randomized variable
Plac ebo
sa mple
Usually a variable that cannot be
controlled – outside of the control of the
experimenter
 Its influence on the dependent variables is
made equal by randomizing into
treatment groups

Treatment
sample
Confounding factors

Factor(s) influencing the results of the
study that are not influencing the groups
in the same way and therefore make the
conclusions uncertain
Confounded variables
Confounded variables vary with the independent variable,
and their effects on a dependent variable cannot be distinguished
from the effects of the independent variable
 Such a relation between two observed variables is termed a
spurious relationship
 Confounding is a major threat to the validity of inferences made
about cause and effect, i.e. internal validity, as the observed
effects should be attributed to the confounder rather than the
independent variable.


Minimize the effect(s) of
confounded variables
Keep them the same and constant for all
groups
 Remove them, e.g. by extraction/purification of
a chemical sample
 Randomize the subjects to the different
treatment groups
 Adjust statistically for the presence of the
confounder(s) when analyzing the data

Makes the study of the independent variable itself more
difficult since more or less of the observed effect may be due to
effect of the confounded variable
Spurious relation

A situation in which measures of two or
more variables are statistically related
(they cover) but are not in fact
causally linked—usually because the
statistical relation is caused by a third
variable.
5
Spurious relation….
Stimulus range and dose-response

If a clearcut relation between the stimulus
(independent factor) and the effect
(dependant factor) can be established, it
increases the probability that the
independent factor is causally related to
the dependent factor(s)
From: http://tylervigen.com/
Choosing levels of the independent variable

Choosing levels of the independent variable
– test wide range and sufficient number
Levels of the independent influence the
results
◦ The stimulus should cover as much of the
range as possible
◦ They should be close enough to prevent
overlooking interesting effects
◦ In within-subjects studies, at least seven
stimuli should be presented if possible
◦ If the continuum is quantitative, the stimuli
should be logarithmically spaced
Choosing levels of the independent
variable- spacing stimuli carefully
III. How do we determine the
correctness of our measurement?

Can you estimate what would be the next
value measured in one of your
experiments? Why/why not?

If anyone else would try to repeat your
experiment, would they get the same
results? Why/why not?
6
Replication
Repeatability and reproducibility
Are others able to replicate/repeat your
results?
 Use replicates to get a better
estimate of the mean and variation

Repeatability - Variation between
results when repeating the same
measurement under the exact same
conditions (within a short time interval)

Reproducibility – Variation between
results when measuring the same thing
(sample) at different conditions (and
times)

A true replicate means a true study of a new
member of the population
o Measuring a characteristic of ONE member of
the population many times is not replication
(but is important to determine the uncertainty
of your method)
o
Controls

Control treatment
◦ Subjects treated in the same manner as the
treated group with the exception of the
substance under study

Sham treatment
◦ Control of the possible effect of a treatment
necessary in addition to the treatment with
the active drug or control. E.g. oophorectomy
operation with and without removing the
ovaries
Random error

Random error can never be completely
eliminated since we can study only a sample
of the population.

Random error can be reduced by the careful
measurement of exposure and outcome

Sampling error can be reduced by optimal
sampling procedures and by increasing
sample size
Random error
The divergence, due to chance alone, of
an observation on a sample from the true
population value
 Leading to lack of precision in the
measurement of association
 Three major sources

◦ Individual biological variation
◦ Sampling variation
◦ Measurement variation
Systematic error = bias
Results that differ systematically from the
true results
 A study with small systematic error has
high trueness
 A study with small systematic and random
error has high accuracy

7
Reliability (of a measurement or of a
study)

The property of a measurement that gives
the same result on different occasions

Related to
(“riktighet”)
(“noggrannhet”)
(“precision”)
◦ Good experimental design
◦ Absence or minimum of confounding factors
◦ Low measurement uncertainty
From: Menditto A, Patriarca M, Magnusson B.
Understanding the meaning of accuracy, trueness and precision. Accred Qual Assur 2007;12:45-47
Validity (of a measurement)
Validity is cumulative
The property of a measurement that tests
what it is supposed to test / measures
what it is supposed to measure
 The best available approximation to the
truth

◦
◦
◦
◦
Internal validity
External validity
Construct validity
Statistical validity
Metrology and statistics


The use of metrological principles ensure that
the measurement techniques used are clearly
understandable, comparable and repeatable
by others
Eurachem/CITAC Guide
http://www.eurachem.org/
Statistics help researchers minimize the
likelihood of reaching an erroneous conclusion
about the relationship between the variables
being studied
8
ISO – quality concepts
ISO – quality concepts
Accuracy
Trueness
True value
Accuracy
(“noggrannhet”)
Precision
Trueness (“riktighet”)
Compliance with the true
Compliance with the true value
value including precision
and accuracy
High accuracy,
Low precision
Precision (“precision”)
Consistency of a series
of determinations
http://www.mathsisfun.com/accuracy-precision.html
The hourglass structure of research
Begin with broad quesions
 Narrow down and focus in on a
particular topic
 Operationalize
 Measure/Observe
 Analyse data
 Reach conclusions
 Generalize back to questions

Low accuracy, but
high precision..!
Fundamentals of good quantitative research






Relevant and intelligent hypothesis
Systematic approach – the researcher processes
logically through steps in a pre- defined plan
Adherence to sound metrological principles
Samples representative of the population =
randomization, stratification etc.
Control – minimizing bias, random variation and
confounding factors
Replication
◦ Repeating observations in your own study
 Most observations where random variation is high
◦ Others try to replicate your results under other conditions
The scientific method is not a recipe

It requires intelligence, imagination, and
creativity.

It is an ongoing cycle, constantly
developing more useful, accurate and
comprehensive models and methods.
9
Take home message
YOU are the one knowing your
experiments best, so
If you have formed a hypothesis
YOU are the one that should make every
effort to try to disprove it
If you can’t do this…
It is likely to be ”true”
Thank you for your attention
and good luck
with your future research!
Acknowledgements to:
Elvar Theodorsson
Jan Gillquist
10