FFIIINNNAAALLL FFIIIEEELLLDDD R
O
R
T
O
F
T
H
E
O
U
N
D
O
F
T
H
E
REEEPPPO
OR
RT
TO
OF
FT
TH
HE
E4
4TTTHHH R
RO
OU
UN
ND
DO
OF
FT
TH
HE
EE
ES
SS
S
Spanish Nacional Team and Metroscopia
March 2009
1.
INTRODUCTION
The European Social Survey is a European study financed by the European Science
Fundation (“Fundación Europea de la Ciencia”) and the European Commission. In
Spain, the main entity financing both the Project and the Coordination and execution
team is the Ministry of Science and Technology. The Galician and Catalan
Governments have also participated in funding this Round of the Survey so that a fully
representative sample for each of these Regions could be obtained.
The Project has very strict technical field work requirements which are supervised by the
National Coordinator, Mariano Torcal, and his team.
The ESS’S main target is to obtain a comparative and longitudinal study of attitudes and
behaviours of European citizens, specially in questions related to social, economical and
political issues.
The Survey is representative of all persons aged 15 and over residing in Spanish
households independently of their Nationality, citizenship, mother tongue or legal status
of residence.
The total size of the sample is of 3.962 cases, including in this number the two
oversamples for Catalonia (905 cases) and Galicia (802 cases)
ESS 08. Final Field Report
Page 1 from 15
2.
FIELD WORK
AND METHODOLOGY
The 4th Round of the ESS has been carried out through in person interviews, as per
the following technical specifications:
•• TARGET POPULATION
The population is composed of all people aged 15 and over
residing within private households in Spain including, Ceuta and Melilla, regardless
of their nationality, citizenship or language.
••
SAMPLING
FRAME
The sampling frame for the 2008 ESS sample is the Spanish
population census structured in sections taken from the Continuous Census
(Padrón Contínuo) updated in January 2008 by the Instituto Nacional de
Estadística (INE, the Public Statistics Office of Spain).
••
SAMPLING DESIGN : A stratified two-stage sample design.
The strata are obtained by crossing two population classification criteria. The first
criterion is the Autonomous resident Community, the Spanish regions (there are 17
of them plus another one grouping the North-African autonomous cities of Ceuta
and Melilla). The second criterion (the type of habitat criterion) distinguishes among
four types of habitat according to their size:
•
The first bracket: cities with more than 100,000 inhabitants aged 15 and over
•
The second bracket: cities between 50,001 and 100,000 inhabitants aged 15
and over
•
The third bracket: municipalities between 10,001 and 50,000 inhabit. aged 15
and over
•
The fourth bracket: municipalities with less than 10,001 inhabitants aged 15 and
over
The cross-tabulation of the two criteria gives a total of 72 theoretical strata (18x4),
only 64 of them being effective. In each stratum the two sampling stages are the
following:
ESS 08. Final Field Report
Page 2 from 15
1. In the first stage, a fixed number of census sections are drawn with probability
proportional to the number of inhabitants of 15 years old and over in each
section. Thus, census sections are the primary sampling units (PSUs)
2. In the second stage, for each PSU selected in the previous stage, 6 or 7
individuals per unit will be randomly drawn: 7 in the sections belonging to the
first two brackets and 6 in the others. The data analysis of previous rounds
showed a response rate in the two first brackets lower than in the rest (62% and
71%, respectively, in 2006, e. g.).
This round received also extra funding from the Galician and Catalonian regional
governments in order to produce a representative in both regions. The probabilities
of inclusion of sections and individuals are provided by the INE
•• INTERVIEW TYPE:
In person interview, administered in the households of the persons
included in the sample, using PDAs (HAPIs), with a structured and pre-codified
questionnaire with some open and semi-open questions. The duration of the
interview has ranged from 45 minutes to 1 hour.
••
DATES OF FIELD WORK: Fieldwork took place from September 5, 2008 to January 31,
2009, except in Galicia which it last till February 19, 2009.
ESS 08. Final Field Report
Page 3 from 15
3.
FIELD WORK
1. DATA COLLECTION
As in ROUND 3, Metroscopia collected data for the ESS through HAPIs (Handheld
Assisted Programmed Interview).
A total amount of 102 devices were used for field work: 72 HP6515 IPAQ from the
previous Round and 30 HP6915 IPAQ purchased specially for this 4th Round.
These machines are very light (165 gr.) and very manageable for the interviewers.
Both the questionnaire and the Contact Forms were programmed in the HAPI, using
the same software from Round 3 which was very much improved, allowing us a much
more thorough and daily control of field work process. Information on both visits and
questionnaires were available at Metroscopia central offices almost on-line. This
allowed us:
A. To keep a closer control of the daily results in production, providing us with
tools to change strategies and relocate teams in order to adapt them to the
work’s evolution
B. To supervise interviews at almost the same time as they were completed,
allowing us to control quality in a very effective way.
2. NUMBER OF INTERVIEWERS
Several restrictions had to be taken into consideration in order to comply with the
project’s specifications about the interviewers:
a. The target (70% response rate) and timings: Approximately 2.660
interviews from a nominal sample of 3.962 households, to be completed in
16 weeks.
ESS 08. Final Field Report
Page 4 from 15
b. The project’s limitation of number of cases completed by each interviewer
(48).
c. The optimal initial number of interviewers which would assist to the main
training, taking into consideration:
••
Cost of training
••
Availability of HAPIs
••
Sufficient number of cases per interviewer so as to motivate them in
terms of expected retributions
Thus, fieldwork started with a total number of 97 interviewers, divided into different
groups which depended on 24 different regional coordinators.
Initially, no more than 48 cases were assigned to each interviewer, and the workload
was proportionally divided between the different regions and interviewers.
At the end of the field process, a total of 138 interviewers had participated in the
project because of the necessity of replacing some of them or reinforcing some
geographical areas with problems.
3. HIRING INTERVIEWERS
Coordinators were requested to hire interviewers for the project responding to a
specific profile:
••
Able to work with HAPI, and familiar with computers
••
Experienced interviewers, whenever possible
••
Available in weekends and late-hours during working days so as to comply with
the project specifications regarding visits during “special hours”
All interviewers were informed about the project requirements concerning times and
days of visits per case.
ESS 08. Final Field Report
Page 5 from 15
4. TRAINING
The main targets in the training processes organized for this project were:
a. Explaining the goals of the Survey
b. Training on the procedures for data retrieval
Most of the time dedicated to training, took place as a guided review of the
questionnaire, with detailed question by question consideration.
4.1. MAIN TRAINING
Main training took place in three different groups:
Madrid, 4th September (64 persons)
Barcelona, 9th September (28 persons)
La Coruña, 10th September (18 persons)
A total number of 110 persons attended this first training.
Main training was conducted by the National ESS coordinator and his team (Mariano
Torcal and Guillem Rico) and technical and field staff in Metroscopia (Adriana
Escardó, Paula López and José Rollón)
4.2. ATTRITION TRAINING
As time elapsed, some interviewers had to leave the project, both for personal
reasons (lack of interest or inadequate commitment with the Survey quality
standards) or because they had reached the maximum of 48 cases established for
this Survey. Therefore, several attrition trainings took place. 28 people received
attrition training.
ESS 08. Final Field Report
Page 6 from 15
Attrition was conducted by Metroscopia Central Office team and the regional
coordinators. Metroscopia’s office staff trained on the relevance and goals of the
Survey. Regional coordinators guided their new team members through the
questionnaire. Then, each new interviewer was requested to make a call to
Metroscopia Field Managers at Metroscopia Central offices in order to be finally
approved to participate in the project.
All 138 interviewers who participated in this Survey received adequate training
(Annex 1)
4.3. PRODUCTION
Metroscopia and the Spanish National Team produced an elaborate set of reports in
order to control and manage fieldwork responding to the ESS special requirements.
A key point on this task was the elaboration of a very detailed projection plan (Annex
2) for the whole of the field period, producing targets per week and province of
completed interviews. An exhaustive control of these initial projections allowed
Metroscopia Field Managers together with the Spanish team to readapt strategies in
order to ensure the capability of complying with the targets (Annex 3).
4.4. PRODUCTIVITY
Production target in the ESS was to achieve a 70% response rate. This had to be
done among a nominal sample of 3.962 cases and allowing no more than a 4% of
ineligibles.
Productivity was high during the first 13 weeks of data collection, with a peak of 336
interviews in week 6 and an average of 193 interviews per week. After week 13,
there was a sudden fall in productivity which brought the average of the last 5 weeks
down to 10 completes per week.
GRAPH 1. FIELD PRODUCTION EVOLUTION. WEEKLY COUNT
ESS 08. Final Field Report
Page 7 from 15
400
Interviews. Weekly
Count
Initial Projections
350
300
250
200
150
100
50
0
1
2
ESS 08. Final Field Report
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20
Page 8 from 15
GRAPH 2. FIELD PRODUCTION EVOLUTION. CUMULATIVE
3.000
2.500
2.000
1.500
1.000
Interviews
500
Initial Projections
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20
Week
Productivity was affected by:
••
“Touched” sample:
Our results for the first phase, weeks 1-8, (2.089 completes) exceeded the
projections (1.851 completes). We obtained a 54,5% “true” response rate and
62,58% “field” response rate.
For the second phase we had to work with a sample where all cases had
received at least one visit attempt. This made it far more difficult to complete
new complete interviews.
••
Geographical dispersion of cases
ESS 08. Final Field Report
Page 9 from 15
Towards the middle of the field work period, the geographical dispersion of the
sample became a bigger issue as it became more difficult to re-visit certain
non completed cases, both in terms of cost and time.
Regional differences made us have to relocate teams on several occasions,
this also affected production during some weeks.
••
Regional differences
As will be further explained, regional differences (dispersion, refusal rates)
produced very different results in terms of productivity
••
Official count of Ineligibles
Field work ended with a final “true” response rate of 67%. This meant taking
into consideration a 3% of ineligibles according to the definition of the CCT in
Europe. “Field” response rate was of 76%.
4.5. CONTACT PROCEDURES
ESS’s technical clauses clearly specify the contact requirements for each case.
These clauses, define two different phases (weeks 1-8 and 9-16) with different
targets to meet:
FIRST PHASE
••
Letters
As per the Survey’s requirements, Metroscopia has sent different rounds of
letters:
A first round of 3.962 letters was sent to the whole sample during August.
A second round was sent to all households approximately 2 weeks before the
interviewers visited each address. This second round, contained the bilingual
letters for Catalonia, Galicia, Navarra and the Basque Country.
ESS 08. Final Field Report
Page 10 from 15
Further mailings were sent, targeting non contacts (798 letters) and refusals (934
letters)
••
Visit Attempts
First phase’s first priority was production. Metroscopia team went out to the field
with a vast virgin sample of 3.962 cases and a goal of bringing as many
completes as possible. The interviewers knew what would allow us to consider a
case as exhausted, but had to prioritise on production.
The rhythm of completed interviews was quicker than expected and we ended the
8th week with a 54,5% of response rate. On the other hand, we only succeeded in
achieving at least one visit attempt to the whole of the 3.962 in the sample during
that same week.
Completing interviews among the new sample cases proved to be easier than we
expected initially, while attempting visits in the whole of the sample was harder
than we thought.
••
Special Hours
The 8th week finished with 31% (1.246) of the cases having received a visit during
a weekend (a total amount of 1.864 of the visits took place during the weekend)
and a 20% of the cases (798) having received a visit during a labour day after
20:00 (in a total of 1.035 visits)
150 non completed cases were exhausted as per the ESS’s specifications.
SECOND PHASE
The second phase started with a different picture than the first phase. The strategy
had to change.
••
Visit Attempts
ESS 08. Final Field Report
Page 11 from 15
Our first priority always was to improve response rate, and we directed
Metroscopia teams to prioritise those cases where we anticipated higher
probabilities of achieving good results:
•••
Cases with only one visit and no contact
•••
Cases with less than 3 visits and poor results (open doors)
•••
Cases not visited during “special hours” (weekends and labour days
after 20:00)
ESS 08. Final Field Report
Page 12 from 15
This was done, taking into consideration the regional differences that had
appeared, in terms of production, which meant we had to focus in:
••
•••
Catalonia
•••
Galicia
•••
Others (Cadiz, Alicante…)
Special Hours
By the end of the study, 38% of the cases (1.488) had received a visit during the
weekend and 28% of the cases (1.090) received visits during labour days after
20.00.
Visits after 20.00 made sense in big cities but were difficult to achieve in rural
areas where the length of a day is different.
While complying with the ESS specifications regarding visit procedures was
always one of our targets, production and quality were the priority.
By the end of production period, there were regions where the response rate was
lower than we had projected. Our main efforts were concentrated in obtaining new
completed cases in these regions rather than exhausting regions with already
high response rates.
4.6. REGIONAL DIFFERENCES
Strong regional differences was a problem during the field period:
a. Catalonia
b. Galicia
c. Rest of Spain
Catalonia and Galicia had an oversample in this round, thus, these two regions
became virtually two separate studies and required big teams to work their cases.
ESS 08. Final Field Report
Page 13 from 15
Besides the special size of both regions’ samples, we detected other issues which
made it difficult to obtain as good results as in the rest of Spain
TABLE 1
REGIONAL DIFFERENCES. RESPONSE RATES
ELIGIBLE SAMPLE
••
••
CATALONIA
GALICIA
•• REST OF SPAIN
•• TOTAL
883
787
2.187
3.857
NUMBER OF
COMPLETES
527
500
1.549
2.576
RESPONSE RATE
59,68%
63,53%
70,83%
66,78%
CATALONIA
During pre-test, we produced poor results in Catalonia. This made us reinforce the
initial projected team for the region. However, production in Catalonia was still
difficult.
Catalonia showed a higher number of refusals, proportionally than the rest of regions
TABLE 2
REGIONAL DIFFERENCES. REFUSALS
REGION
••
Cases which received at
least one refusal
••
% within Region
••
Total Count
TOTAL
REST OF
SPAIN
GALICIA
CATALONIA
401
161
251
813
17,78%
20,07%
27,73%
20,52%
2.255
802
905
3.962
GALICIA
The reasons behind the difficulties in completing cases in Galicia were different.
ESS 08. Final Field Report
Page 14 from 15
••
GEOGRAPHICAL
DISPERSION:
Galicia is a rural area with many small towns and
villages. Interviewers needed to attend to most of their visits by car
••
INE’S SAMPLE: addresses in Galicia are expressed with a much more complicated
layout than in the rest of Spain (parroquia, ) The initial data we received from INE
was more difficult to understand and addresses more difficult to find
ESS 08. Final Field Report
Page 15 from 15
5. NON CONTACTS AND REFUSALS
While during Phase 1 field team’s priority was to sweep the whole sample almost
blindly in order to get as many completes as possible, Phase 2 required a much finer
strategy. Field managers started studying the sample in a case by case basis,
directing Metroscopia teams under carefully studied plans.
5.1. NON-CONTACTS
Central Office staff produced daily reports to control the situation of the sample.
Cases showing non-contacts where paid special attention. At the end of week 11 we
still had 217 cases which had been visited but never contacted. These, to us,
represented a special opportunity. Metroscopia interviewers received lists of these
cases in their regions, prioritising cases that had received less than 4 visits (142 at
the end of week 11)
By the end of data collection, only 94 cases had never been contacted*. The average
number of visits received in these cases was of 5,8, and only 9 of them had received
less than 4 visits.
* The general reports that are requested in the ESS, regard the situation of a case
only by the result of the last visit received. This, in the case of the non-contacts may
lead to errors. We have separated our analysis and studied with a special focus
those cases that have never received any contact. Cases where only the last visit is
a non contact (up to 232) are studied taking into consideration the results of prior
visits.
5.2.
REFUSAL CONVERSION
All cases that received a refusal were marked as “still in sample”. As field work
elapsed, our analysis detected that there were certain regions where a low response
rate could be caused by a high refusal rate: Catalonia.
As shown in Table 2,
Catalonia showed a much higher rate of cases that received refusals.
ESS 08. Final Field Report
Page 16 from 15
Therefore, Catalonia had to be tackled in a special way. As per the ESS’s
specifications, this meant making a bigger effort in refusal conversion, via sending
new “converting interviewers” (more experienced)
ESS 08. Final Field Report
Page 17 from 15
Table 3 clearly shows how much bigger the effort was in Catalonia. In it, we see how
many “converting interviewers” visited each case that received a refusal, taking into
account that each interviewer who tried to convert more than one case is counted
more than once.
Thus, in Catalonia every refusal received an average of 65,74% converting
interviewers, compared to the
16,96% of the rest of Spain. However, with a
significantly bigger effort, we only obtained the same rate of conversion in Catalonia
than in the rest of Spain. This illustrates well why, despite the efforts in the region,
the response rate in Catalonia ended way behind the response rate in the rest of
Spain.
It is also significant that a similar pattern appears in Galicia.
There were more
refusals than in the rest of Spain, despite the bigger effort to convert cases and
poorer results in general.
TABLE 3
EFFORTS IN REFUSAL CONVERSION
REGION
REST OF
SPAIN
GALICIA
CATALONIA
TOTAL
••
Cases which received at
least one refusal
401
161
251
813
••
Number of interviewers
that tried to convert
refusals (the same
interviewer is counted
more than once if he/she
has tried to convert more
than one case)
68
75
165
308
16,96%
46,58%
65,74%
37,88%
103
31
63
197
25,69%
19,25%
25,10%
24,23%
••
% of converting
interviewers per refusal
••
Refusal conversion. Count
••
% of Conversions
ESS 08. Final Field Report
Page 18 from 15
6. SUPERVISION
6.1. NON CONTACTS AND REFUSALS
Exhausting all possibilities among cases that didn’t end up in a complete interview
appeared as a very important task to undertake since we had to work with a nominal
sample and obtain from it as many interviews as possible. From the beginning, we
organized our teams so that a second interviewer would supervise most of the cases
where we could not obtain an interview.
All cases that finished as a non contact or a refusal received, at least, 4 visits.
From these, a 46,8% of the total count of non contacts received a visit from a second
interviewer.
A 39,6% of all cases that received a refusal, were visited by at least one second
interviewer.
TABLE 4
SUPERVISION
••
••
Supervised (2nd interviewer)
Total Count
•• Percentage
NON CONTACTS
(V5=6; V5=7 & V12 = 1-7)
REFUSALS
(V6=2; V6=3-4)
141
301
46,8%
210
530
39,6%
6.2. VALIDATING THAT INTERVIEWS HAVE BEEN DONE
Through an automatic process, the system randomly chose a 15% of the interviewed
cases to be validated. These households received a call from Metroscopia offices and
were asked the following questions:
ESS 08. Final Field Report
Page 19 from 15
VALIDATION FORMS
••
Did the interview take place?
•• Did you receive a brochure with information about the
Project?
•• Did the interviewer give you an incentive?
••
••
••
••
What did this incentive consist of?
Did the interviewer display show cards for certain
questions?
Do you recall the interview length? [does this resemble the
real length?]
Place where the interview took place
YES/NO
YES/NO
YES/NO
GIFT CARD/OTHER
YES/NO
YES/NO
HOUSE/OTHER
6.2. SUPERVISING QUALITY IN THE INTERVIEWS
Quality control was carried out in parallel to field work. Having the information from
the interviews at Metroscopia Central Offices on-line made this possible.
••
AUTOMATIC
FILTERS:
The system introduced a program to automatically locate a
number of pre-designed inconsistencies. (Anex 4) These inconsistencies were
marked as “impossibles” or “improbables”. Interviews showing more than 2
“impossibles” or more than 10 “improbables” were automatically selected for
supervision.
••
SUPERVISING
INTERVIEWERS’ WORK:
the first interview of each interviewer and, at
least a 10% of his/her cases were automatically selected for supervision. This was
done in parallel to field work and it allowed us to correct issues and solve
problems in good time. This, actually, became another way of permanent training.
••
SUPERVISING
AN INTERVIEW:
qualified staff studied these automatically selected
interviews to detect issues. They would call those households where the interview
showed weird patterns in order to correct them.
By the end of the project, more than a 27% of the total completed cases had been
supervised.
ESS 08. Final Field Report
Page 20 from 15
Quality has been a very important goal.
7. CONCLUSION
The Fourth Round of the ESS has been completed with very satisfactory results.
FINAL SUMMARY
••
UNITS CONTACT ATTEMPTED
3.962
••
ACHIEVED COMPLETE INTERVIEWS
2.576
••
INCOMPLETE INTERVIEWS
••
REFUSALS
527
••
NON-CONTACTS
232
••
UNAVAILABLES – NOT TRACEABLE
510
••
“TRUE” INELIGIBLES
105
••
APPOINTMENTS PENDING
3
9
Improvements in the software have allowed us to keep a much closer control of
fieldwork evolution, especially because the Contact Forms was also programmed in
the HAPI’s
Field work management took place with detailed information that allowed the staff to
make quick decisions and adapt their strategies to each situation.
This Round of the Survey has allowed us to concentrate more in better training the
interviewers. This and the fact that we have continually insisted in refreshing this
training (via attrition and supervision) have produced good results in terms of quality.
ESS 08. Final Field Report
Page 21 from 15