ASSESSING AND COMPARING
QUALITATIVE AND QUANTITATIVE METHODS
This is based on an exercise that was done as part of workshop for CARE Georgia staff
and was facilitated by Mary Picard, the Regional Program Advisor for MERMU.
Participants were given a set of 15 statements which they then had to rank as true or
false. Each statement was ranked by individuals (and not groups) on individual pieces
of paer, one at a time and the results then aggregated. The purpose of this exercise
was not to test their level of expertise but to see where they stood on the differences
and value of quantitative vs. qualitative methods. Some participants answered with a
question mark, so a column was added for “don’t know.” The anticipated “correct”
answer for each statement is in the column “ans.” The actual results of the exercise as
done in CARE Georgia are not given; columns are left blank for use by others who wish
to try out this exercise.
NO STATEMENTS
1
Less tangible impacts are less important than
economic impacts
2
If it can’t be quantified, it can’t be validated.
3
Qualitative means “subjective” and
quantitative means “objective”
4
Quantitative is more rigorous and precise
than qualitative methods
5
Quantitative is scientific and qualitative is
not.
6
Qualitative data can not be quantified.
ANS.
F
7
F
8
9
10
11
12
13
14
Qualitative methods are the same as
participatory methods.
Donors prefer quantitative results over
qualitative.
Qualitative methods require a higher skill
level than quantitative methods.
A survey is a quantitative method.
The use of qualitative methods does not
require sampling.
All qualitative data are anecdotal.
Qualitative methods allow beneficiaries to get
involved in the process of monitoring change
more easily than quantitative.
The use of qualitative methods for data
collection is more time-consuming than the
use of quantitative.
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
#F
#T
?
F
F
F
F
F
?
F
F
F
F
T
?
1
NO STATEMENTS
15 Participatory methods are only able to
generate data if on a small scale.
ANS.
F
#F
#T
?
ANS.
F
#F
#T
?
Explanations of each statement follow.
NO STATEMENTS
1
Less tangible impacts are less important than
economic impacts
“Less tangible impacts” could refer to civil society activities as discussed by the CARE
Georgia staff. These activities do not generally produce quantifiable impacts, such as
increases in income, # of jobs created, or tons of crop yields. Their measures are
highly qualitative and often based on perception and on relationships between people –
levels of collaboration or cooperation, improvements in accountability and participation,
empowerment of vulnerable groups. Just because they are neither describable in
numbers nor concrete does not mean they are any less important than tangible
impacts.
NO STATEMENTS
2
If it can’t be quantified, it can’t be validated.
ANS.
F
#F
#T
?
This is similar to the first statement. It relates to the validity of qualitative data when it
is expressed verbally and not numerically. How valid the data are depends on how well
the method was applied (quantitative or qualitative), the skill of the data collector, how
well the process was documented, the quality of the analysis and interpretation. Thus,
qualitative data are no less and no more valid than quantitative data, nor are
quantitative methods more or less prone to validation than qualitative methods.
NO STATEMENTS
3
Qualitative means “subjective” and
quantitative means “objective”
ANS.
F
#F
#T
?
In one sense, qualitative is associated with the subjective and quantitative with the
objective, however, this is not a fair dichotomy.
The difference lies in the school of thought behind each approach. The school of
thought associated with quantitative methods is referred to as ‘logical positivism.’
Paraphrasing from Christiaensen: 1
1
Christiaensen, Luc. 2001. “The Qual-Quan Debate Within Its Epistemological Context: Some Practical
Implications.” Workshop held at Cornell University, 15-16 March 2001. Qualitative and Quantitative Poverty
Appraisal: Complementarities, Tensions, and the Way Forward.
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
2
In the logical positivist school, there exists a single, external reality and it is the
analyst's task to capture this as closely as possible. To do so, the analyst seeks
to increase the likelihood of unbiased, objective answers mainly by relying on
statistical principles in its study design (experimental, quasi-experimental,
representative sampling) and structure, standardization and quantification in its
data collection. These principles are intended to guarantee the representativity
that would permit a generalization of the results for the population under study.
Thus, the analyst's role is the provision of objective information to the decision
makers.
Qualitative research methods on the other hand are associated with the
interpretivist and the constructivist tradition. This school of thought starts with a
recognition of a multitude of realities and proposes that objectivity and valuefree science are simply impossible (House, 1994 and Hedrick, 1994 quoted in
Cristiaensen, 2001). To fully understand the topic of interest within its context,
the inquiry methods used seek to involve many stakeholders and to obtain
multiple perspectives on the subject of research and the meaning of the
concepts, through semi-or unstructured, exploratory data collection methods. In
the constructivist tradition, the analyst does not only aim to provide and facilitate
an understanding of the subject, but also seeks to bring about change and
empowerment of the stakeholders in the process.
In sum, to explore any particular subject, quantitative methods implies different
assumptions about reality than quantitative methods, but it does not mean the methods
can not be combined.
NO STATEMENTS
4
Quantitative is more rigorous and precise
than qualitative methods
ANS.
F
#F
#T
?
This is a myth. Both methodological approaches are well established in their field. To
wit, the real test of rigour and precision is in the application and the user of the
methods. Neither method is inherently more or less rigorous or precise.
NO STATEMENTS
5
Quantitative is scientific and qualitative is
not.
ANS.
F
#F
#T
?
This too is a myth. Quantitative methods are well-established in the natural sciences
and the discipline of economics, and qualitative methods are a well-established
academic tradition in the social sciences (such as sociology and anthropology).
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
3
NO STATEMENTS
6
Qualitative data can not be quantified.
ANS.
F
#F
#T
?
The use of qualitative methods can generate numerical data through the use of ranking
and scoring, for example. It is possible to count, average, calculate the number of
people doing different things, the number of people with a common perception, or
quantities of things being produced by people, among others.
NO STATEMENTS
7
Qualitative methods are the same as
participatory methods.
ANS.
F
#F
#T
?
Qualitative methods can either be participatory or non-participatory. The technique of
observation, for example, is not participatory. Qualitative methods, broadly speaking,
leave the researcher with the choice of how to investigate the issues and with the
analysis, rather than participants (beneficiaries) determining the process. Another key
difference between qualitative and participatory methods is that qualitative methods
seek to understand what is happening rather than to influence events or empower
people.
NO STATEMENTS
8
Donors prefer quantitative results over
qualitative.
ANS.
?
#F
#T
?
This has no right or wrong answer. Generally, in my (the facilitator’s) experience,
donors do prefer and are accustomed to seeing quantitative results. Including this
statement was a way to find out what the experience of participants was on this score.
NO STATEMENTS
9
Qualitative methods require a higher skill
level than quantitative methods.
ANS.
F
#F
#T
?
This is not necessarily true; it depends on the exact method and on the skill set of the
staff person or researcher. Amongst the array of quantitative methods, focus group
discussions, for example requires a high skill level. The issue is that many people are
intimidated by qualitative methods because they do not have the training nor adequate
experience in using them.
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
4
NO STATEMENTS
10 A survey is a quantitative method.
ANS.
F
#F
#T
?
Depending on the types of questions asked in the survey, it can be quantitative or
qualitative. Open-ended questions are a form of qualitative methods and structured
questions – multiple choice or yes/no – are quantitative.
NO STATEMENTS
11 The use of qualitative methods does not
require sampling.
ANS.
F
#F
#T
?
Selecting the population for a study, whether using qualitative or quantitative methods,
usually involves some form of sampling, unless the population with which a project is
working is very small (and you can interview all participants). Thus, to select
participants for focus group discussions, case studies, qualitative surveys, key informant
interviews or other qualitative methods still may require sampling the population.
However, qualitative methods often, though not always, require a different kind of
sampling called purposive sampling.
Purposive sampling intentionally chooses respondents for a specific reason, e.g., girls
attending primary school or drug users with HIV/AIDS. When using this type of
sampling, it is very important to define the selection criteria, i.e., to know who you
must include and why. This is because purposive sampling is used in association with
developing information-rich cases for in-depth analysis related to the central issues
being studied (Barton, 1997). 2
Purposive sampling is selective sampling and is not based on statistical probability used
in quantitative studies. Probability sampling depends on randomness as a way to
confidently generalize results for a small sample to a large population (Barton).
Purposive sampling can be used either with qualitative or quantitative methods.
NO STATEMENTS
12 All qualitative data are anecdotal.
ANS.
F
#F
#T
?
Qualitative research, if done well, is intended to avoid anecdotal reporting. “Anecdotal’
is ad hoc, occasional, or in this context, information that is gotten in passing or in an
unplanned fashion. For example, a project staff person goes to the field and walking
through the village, meets a woman who tells him about the benefits of the new water
system. This is different from going to the village with a questionnaire or interview
guide that will be used with a sample of the population. Anecdotal information can be
useful and valid but should not be used to generalize to a larger population.
2
Barton, Tom. 1997. “CARE-Uganda. Guidelines to Monitoring and Evaluation: How Are We Doing?”
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
5
NO STATEMENTS
13 Qualitative methods allow beneficiaries to get
involved in the process of monitoring change
more easily than quantitative.
ANS.
T
#F
#T
?
Compared with quantitative methods, this is a true statement. Qualitative methods
generally require in-depth face-to-face field work that puts researchers or the
interviewers in the context being studied in order to examine the context holistically.
This is different than sending out unskilled enumerators to fill a questionnaire.
Qualitative methods which encompass forms of participatory inquiry more intentionally
engage participants or beneficiaries in monitoring change.
NO STATEMENTS
14 The use of qualitative methods for data
collection is more time-consuming than the
use of quantitative.
ANS.
?
#F
#T
?
This is neither true or false. It depends on which qualitative or quantitative methods
are being applied. What is time-consuming about qualitative methods is often the
analysis and synthesis of the data. Aggregating verbal information (reading transcripts
of interviews, scanning text for patterns) does take more time than adding up numbers.
The answer is it is difficult to say and depends on many different things, such as the
skill of the researcher, the specific type of method used, the quality of the data
gathered, and how much analysis and aggregation are necessary.
NO STATEMENTS
15 Participatory methods are only able to
generate data if applied on a small scale.
ANS.
F
#F
#T
?
Participatory methods are not limited to any scale, geographic or population size.
Participatory rapid appraisal tools for example have been used to assess poverty at the
national level. Especially when participatory methods used are able to generate tables
of data, there is no limit to the scale of their application. More and more examples are
available whereby participatory methods are being used in lieu of questionnaire surveys
on a large scale (Chambers, 2003). 3
3
Chambers, Robert. 2003. “Participation and Numbers.” PLA Notes 47 (August).
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
6
Some Broad Differences Between Qualitative and Quantitative Methods:
QUALITATIVE
QUANTITATIVE
Seeks to capture the complexity of
issues in their interrelatedness rather
than discrete variables (a holistic
approach)
Separates and simplifies processes into
variables for analysis purposes
Used to give depth to findings, usually
by focusing on information from
individuals
Looks at trends and prevalences within
a population area (breadth of
information)
Largely answers the questions “why”
and “who”
Largely answers the questions “what,”
“how many,” and “for whom”
Based on a school of thought that
recognizes a multitude of realities
Based on a school of thought that
assumes a single, objective reality
Often inductive process of inquiry –
discovering, letting issues evolve,
looking for patterns
Mostly deductive – based on predetermined questions and hypotheses
Information generated expressed
mostly verbally or visually
Information generated expressed
mostly in numbers (proportions and
percentages)
Exercise based on a CARE Georgia Workshop
Facilitated by Mary Picard, MERMU Regional Program Advisor
7