Assuring good field work
Juan Muñoz
What happens when fieldwork is poor?
• A long and frustrating process of “data cleaning”
becomes unavoidable
The data loose their policy-making relevance
• Data quality is not guaranteed
The process converges (at best) to databases
that are internally consistent
• The process entails a myriad of decisions,
generally undocumented
Users mistrust the data
Key factors
• Manage the survey as an integrated
project
• Implement the team concept in the
organization of field operations
• Integrate computer-based quality controls
to field operations
• Establish strong supervision procedures
• Ensure sufficient training
• Work with a reduced staff over an
extended period of data collection
Management levels
• Core staff
– Survey manager
– Field operations manager
– Data manager
• Tactical options for the organization
of field teams
– Mobile teams with fixed data entry
– Mobile teams with integrated data entry
– Sometime in the future: the paperless
interview
Mobile teams with fixed
data entry
•
•
•
•
•
•
•
Cote d’Ivoire (1984)
Peru (1985)
Ghana
Pakistan
Guinea-Conakry
Mozambique
Iraq (2006)
Composition of a field team
Supervisor
Interviewers
Anthropo
-metrist
Data entry
operator
The team and its tools
Supervisor
Interviewers
Anthropo- Data entry
operator
metrist
Two PSUs visited in a fourweek period
Alama
Bamako
Regional
Office
First week
Alama
Bamako
Regional
Office
They complete
first half of
questionnaires
in all selected
households
Operator
remains in
Regional Office
Rest of the
team travels
to Alama
Second week
Alama
Bamako
Regional
Office
Operator enters
first week data
from Alama
Rest of the
team travels
to Bamako
They complete
first half of
questionnaires
in all selected
households
Second week
Alama
Bamako
Regional
Office
Rest of the
team travels
to Bamako
and back
Supervisor
gives Bamako
questionnaires
to DEO. DEO
gives back
Alama
questionnaires
with flagged
inconsistencies
Third week
Alama
Bamako
Regional
Office
Team completes
second half of
questionnaires.
They correct
inconsistencies
from first half
Operator enters
first week data
from Bamako
Fourth week
Alama
Bamako
Regional
Office
Operator enters
second week data from
Alama. Corrects
inconsistencies from
first round
Team completes
second half of
questionnaires.
They correct
inconsistencies
from first half
Fourth
week
The result is
a clean
data set on diskette,
ready for analysis
immediately after data
collection
Regional
Office
Mobile teams with
integrated data entry
•
•
•
•
Nepal (1992 and 2001)
Argentina
Paraguay
Bangladesh (2000)
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Team works with
portable computers
and printers
Regional
Office
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Operator travels
with the rest of the
field team
Regional
Office
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Data entry and
validation almost
immediate
Regional
Office
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Reduced trips to and
from Regional Office
to selected PSUs
Regional
Office
Mobile teams with integrated data entry
Bamako
Cocody
Alama
Regional
Office
Benefits of integration
• Provides reliable and timely databases
• Provides immediate feedback on the
performance of the field staff, allowing early
detection of inadequate behaviors
• Ensures that all field staff applies uniform
criteria throughout the full period of data
collection
• Solves inconsistencies through direct
verification of households reality, rather that
through office guessing
• Is consistent with the total quality culture
Selecting and training field
staff
• Why is it important
• How long does it take
• How is it organized
Example: Day 2 of interviewer training
• Definition of household (and dwelling, family, etc.)
• Pictorial of a sample household
• Slide with an empty roster (explain case
conventions, encodings, skip patterns, etc.)
• Fill the roster for the sample household (need for
legible handwriting, recording of ages, use of a
calendar of events, etc.)
• Role playing (trainer as a respondent, simulating
borderline cases)
• Role playing (trainees interview each other)
The role of the team supervisor
• Manager/administrator ("traditional" role)
– Monitor completion of work
– Collecting and accounting for all of the
questionnaires
– Paying interviewers/managing the fuel budget
– Administrative functions
– Sometime interviewer
• Quality control
– Continuous training of interviewers
– Random quality checks in the field
Supervision tasks
• Verification of questionnaires for
completeness
– Completion of household roster & ID of
members
– Completion of all sections for all individuals
– Limited internal consistency
•
•
•
•
Random re-interviews of households
Observation of interviews
Observation of anthropometrics
Supervision of data entry
The paperless interview (CAPI)
• The option of the future
• Is used successfully by some statistical agencies for
simple surveys (LFS and CPI price collection)
• Recent experiments have shown that
– Technology is already available
(Lightweight notebooks and software development platform –
both Windows based)
– Can be cost-effective
– No negative serious externalities
• We still need to solve:
–
–
–
–
Questionnaire design
Ergonomic aspects of the interview
Interviewer training
Development of supervision procedures adapted to the new
technology (voice recording, use of GPS’s, etc.)
Case study: The IHSES
Iraq Household Socio-Economic Survey
Presenter: Shwan J Fatah – Sulaimania, KRG Stat Office
• Each cluster (6 households) visited by one interviewer in a 20day period (a wave)
• Each household records food expenses in a diary for 10 days
• The interviewer visits each household seven times, before
during and after the 10-day diary recording period
• During these visits, the interviewer
– Helps with diary recording
– Asks different questionnaire modules (education, heath, labor, etc.)
– Checks for inconsistencies in the data collected in previous visits
Case study: The IHSES
Iraq Household Socio-Economic Survey
(continued)
• This was possible by organizing the field workers into
teams, composed of
– One supervisor
– Three interviewers
– One data entry operator
• Data was entered and checked in between interviewer
visits
• Fieldwork concluded in January 2008
• A database is already available
• Preliminary outputs expected in March 2008