SHARE Pretest 2010
Feedback & Results
1
Schedule
Date
1. Generic
2. SMS and
Questionnaire data transfer
Design,
Programming
, Testing
3. CountrySpecific
Questionnaire
Design,
Translation
Programming
, Testing
4. Agencies,
fieldwork
prep,
contracts
20 – 21
Jul 2010
5. Fieldwork
7.
Conferences,
Meetings
8. Deadlines
Post-pretest
meeting in
Odense
01 – 15
Aug 2010
Area Teams,
Country Teams
revise generic
and countryspecific
instrument on
paper (->
version 5)
MEA,
CentERdata
revise SMS (->
Version 5)
15 – 31
Aug 2010
CentERdata
CentERdata
programs
programs SMS
Generic Version Version 5
5
01 – 15
Sep 2010
MEA, Area
Teams,
Country Teams
test Generic
Version 5
MEA, Area
Teams,
Country Teams
test SMS
Version 5 (->
Version 6)
16 – 30
Sep 2010
CentERdata
programs CAPI
Final Generic
Version 6
CentERdata
programs SMS
Final Generic
Version 6
Area Teams,
Country Teams
revise generic
and countryspecific
instrument on
paper
MEA finishes
preload
database for
main survey
Survey
agencies test
preload
database for
main survey
Generic Version
6 frozen for
Main Survey
Training
2
Schedule
01 – 07
Oct 2010
Country teams
translate final
changes to
CAPI & SMS in
LMU
08 – 15
Oct 2010
CentERdata
compiles
translated
instruments
16 – 22
Oct 2010
Country-teams
test,
implement final
changes in
LMU;
CentERdata
compiles final
translated
instruments
22 Oct
2010
Final
Translated
CAPI and SMS
Version 6
27 – 28
Oct 2010
02 Nov
2010 - 30
Jun 2011
TTT in
Germany
Fourth wave of
SHARE
-- Everybody
works hard
3
What we've done
since Paris
• Further testing, revision and one (major) change of the
instrument
– Sample Management System
– Translation
• IT meeting for agencies (Apr 23)
– Concepts, ex ante v ex post harmonization
• TTT for new and old countries (May 25-26)
–
–
–
–
–
–
Organized and conducted by MEA
Mock interviews of new modules/questions
SMS
Preload
Gaining respondent cooperation
Proxy etc.
4
Pretest goals
• 100 realized household interviews
– at least 50 refresher households
• Test logistics in all phases of the fieldwork
– includes proxy interviews if necessary
– includes exit interviews where necessary
– included interviews in institutions / nursing homes
where necessary
– have drop-off for each respondent
5
Pretest goals
• Testing the CAPI instrument
–
–
–
–
Do the structural changes to the CAPI work?
In particular new/much changed sections SN and CH
Is the translation/adaptation correct?
Where are we in terms of length?
• Testing the functionality:
– SMS Client
– SMS Client
– SMS SD



CAPI instrument
SMS Sample Distributor
CentERdata
• Get ready for the main survey!
6
Qualitative Feedback
• All feedback received until July 18
(Sunday)
• 7 agencies, 6 countries
– DK, ES, EE, FR, BN, IT
– also received by now: CH, BF, NL, HU
• Going to forward to area teams
– please deal with detailed comments
7
Qualitative Feedback
• Gaining R cooperation
– getting harder and harder to convince people to
participate
– length of interview does not help
– reluctance to answer personal questions to people
living in their community (IT)
8
Qualitative Feedback
• Particular problems to gain cooperation
– older R (afraid, easily tired)
– working R (too busy)
– R living in appartment buildings (difficulty getting to
door)
– earlier refusals (i.e. who did refuse SHARELIFE) hard
to convince to be interviewed again
– couples (two interviews are just too much in one
evening session)
– older Rs often first asked children whether they
should participate --> special info aimed at children?
9
Qualitative Feedback
• General complaints about length, repetitiveness
and detail
– tiredness
– lack of interest
– anger
10
Qualitative Feedback
• Specific length problems identified
– Introductions are too long, too much blablabla (sic!) time robbers (DK, FR)
– Smoother way to do "all proxy interview" needed
– Mental health module too long
– second person refused because of length
• Brochures
– helpful but should be more neutral (i.e. not linking low
education and obesity)
– third wave results needed
11
Qualitative Feedback
• Preload errors
– year of last interview
• What was lacking
– questions about personal interests and activities
• Unfolding bracket entry points were unrealistic
(FR)
• A comprehensive QxQ should be available
before the data collection
12
Qualitative Feedback
• SN:
– too early in the interview, resp do not feel comfortable
– reluctance to give names/details of friends and relatives without
their permission (interview should be about the R)
– people seem to forget people they live with
– better shift after CF because it makes it easier to separate
couple Rs
– hard to separate couples right at beginning of the interview when
insisted both should be present
• CH:
– becomes confusing if too many children with similar
names/nicknames (ES)
13
Qualitative Feedback
• DO
– respondents, especially old did not like filling in (ES)
– between 10 and 30 minutes (DK)
– favorable reaction to have health care questions
(costs) on paper rather than CAPI
– people got annoyed, questions seemed to be already
administered
– people did not understand why they had to fill in DO
after such a long interview
– illiterate respondents
14
Qualitative Feedback
• Measurements:
– are generally liked because they bring variation to the
interview (BN)
– 3 test are too many (IT)
– why can't interviewers choose when to perform test
(like in SHARELIFE)
– The preferred section amongst several respondents,
they were very cooperative and interested during the
physical tests
15
Qualitative Feedback
• Financial questions:
– too difficult, too detailed, too nosy!!! (sic!)
– questions are too long; easier to understand when
you read them than when you hear them
– respondents do not understand what is the purpose of
questions
16
Data on which numbers
are based
• All data received until July 20 (yesterday)
• 12 countries
– no data yet from DE, GR, HU, PT, LU(?)
• 844 household interviews
• Target was 100 hh i'views per country
(reached in only 4 countries so far)
17
Number of interviews
by country
140
120
100
80
60
40
20
0
AT
BE
CH
CZ
DK
EE
Single
ES
FR
IT
NL
SE
SI
Couple
18
Length
• Excluding all interviews shorter than 20 minutes
and longer than 5 hours, the overall average and
median length are:
– 80 and 76 minutes for single interview, 83 and 79 if
weighted
– 130 and 125 minutes for couple interview, 130 and
125 if weighted
• Note: this is pure interview time including
interviewer observations but excluding general
setup and coverscreen
19
Length
• The averages do not tell us much about the
variance
– 35% of the total variance is between countries
– as always, there are "long" and "short" countries
20
0
50
100
150
200
Length by country,
single hh
CH
NL
AT
FR
EE
BE
SE
DK
ES
CZ
IT
SI
excludes outside values
21
50
100
150
200
250
300
Length by country,
couple hh
CH
DK
SE
AT
NL
BE
FR
EE
ES
CZ
SI
IT
excludes outside values
22
Length (Pretest v Main)
• Comparison of wave 2 timings between
pretest and main survey
– Single HH took 1.3 minutes longer in main
than pretest
– Couple HH took 4.5 minutes longer in main
than pretest
• Changes between main and pretest?
– Added Chair Stand and Peak Flow!
– Added CASP and CES-D to AC module
– Various minor cuts and additions
23
Length (Pretest v Main)
• Accounting for CS, PF, AC additions
– Single HH: 5.9 minutes shorter
– Couple HH: 8.3 minutes shorter
• Yet again: huge between-country variance
24
-4 0
-2 0
0
20
40
Pretest v Main
(single HH - wave 2)
bf
at
de
dk
pl
it
bn
Country
es
ch
nl
se
gr
25
Length by area
• Single / couple hh
– Health:
– Econ:
– Social:
34.8 / 65.6 min
23.6 / 42.6 min
19.8 / 32.0 min
• Note: allocation of modules to areas does
not allow for overlaps
– e.g. CF, DN, HO
26
0
5
10
15
20
Length by module
CF EP PH EX HC HO AC MH SN CH BR AS WS GS DN CO PF CS SP FT IV HH
excludes outside values
27
If to cut, where to cut?
• Expecting a too long interview, we have discussed
in May with area teams a contigency plan
– two round of discussion
– designed to take on average 10 minutes off the CAPI
questionnaire
• achieved reasonable degree of agreement on
what to cut
• scheduled some of the dropped item for next wave
/ in-between data collection
28
If to cut, where to cut?
• Health
– Drop Walking Speed/Chair stand
– Skip HC including end of life care – with the exception
of
•
•
•
•
HC002 (seen a doctor in last 12 months),
HC003 (seen a GP in last 12 months),
HC004 (seen a specialist in last 12 months),
HC005 (which specialist), HC012 (been in Hospital in the last 12
months),
• HC013 (How often in Hospital in last 12 months),
• HC014 (total nights in hospital in last 6 months),
• HC029 (been in nursing home last 12 months), and the
– Health insurance questions in drop-off?
29
If to cut, where to cut?
• Social
– Drop year of birth from SN loop
– FT: drop FT004, FT006, FT011, FT013, FT015 etc. large
gifts or inheritances (amount and reason for transfer)
– HO071, HO072, HO073: Quality of neighborhood
– SP: return to the single solicitation of help
given/received outside the household (i.e. do not
distinguish between personal care and practical help)
30
If to cut, where to cut?
• Econ
– EP021, EP022, EP057, EP058: Responsibility for
supervising employees; how many
– EP059: Opportunities to work after retirement age
– EP065: Retirement been a relief or concern
– HO035: Years living in community
– HO038, HO039, HO040: Apart from vacations or brief
visits, do you regularly spend part of the year in another
residence? Where is this residence located? In Which
country
– HO011, HO012: ask only at baseline or if moved house
since last wave...
31
If to cut, where to cut?
• Econ
– Drop all asset income question (AS005, AS009,
AS015, AS058) – replace by summary question
– AS061 reason no bank account  only at
baseline
– Keep expectations module only for baseline
– Drop financial literacy questions from EX
module
– Drop detailed ISCO questions
32
If to cut, where to cut?
• Scheduled for next wave / in-between
wave data collection:
– End of life decisions
– Quality of living environment
– Prevention
– Financial literacy
– Walking speed / Chair Stand / Other
measurement?
33
Questions?
?
34