The Development of a Successful
Household Panel Survey:
The HILDA Experience
Mark Wooden
Project Director, HILDA Survey
About HILDA: Introduction
 Funded and owned by Australian Government
 Multi-purpose survey
– Modelled on other household panels – BHPS, SOEP
 Survey manager = Melbourne Institute of Applied
Economic & Social Research (University of Melbourne)
 Fieldwork subcontractor = Roy Morgan Research
 Unit record data available (under license)
 Want to know more?
 Articles in The Economic Record, June 2007 and Australian
Economic Review, September 2010
 www.melbourneinstitute.com/hilda/
www.melbourneinstitute.com
About HILDA: Key Design Features
 Commenced (in 2001) with national probability sample of
households
– Area-based clustered / stratified sample design




Annual survey waves
Follow all original hh members and offspring indefinitely
Sample augmented with hh joiners
Interview all “adults”
– Face-to-face where possible
– CAPI / CATI technology
 Refreshment (top-up) sample added in wave 11
 Cash incentives paid
www.melbourneinstitute.com
Survey Instruments
 Household Form
– Key identifiers / Changing HH membership / HH relationships /
Reasons for non-response
 Household Questionnaire
– Collects hh level data from relevant HH member
 Continuing Person Questionnaire
– All persons 15+ who have previously been interviewed
 New Person Questionnaire
– All persons 15+ who have never previously been interviewed
 Self-completion Questionnaire
– All interview respondents; 16 pp, expanded to 20 from W5
www.melbourneinstitute.com
What’s In It? HQ / CPQ
 Core:
–
–
–
–
–
–
Child care
Housing
Education
Employment status
Job characteristics
Job search
–
–
–
–
–
Calendar
Income
Family formation
Partnering & relationships
Living in Australia
• Disability, Life satisfaction,
Spatial mobility, Caring
– Tracking
– Interview situation
 Special “modules”:
–
–
–
–
–
–
–
–
–
–
–
–
W1 (+NPQ) = Personal history
W2 = Wealth
W3 = Retirement
W4 = Youth issues; Private
health insurance
W5 = Family formation
W6 = Wealth
W7 = Retirement; Lifestyle
W8 = Family formation; Noncores. relationships
W9 = Health
W10 = Wealth
W11 = Family formation;
Retirement
W12 = Skills & abilities; Noncores. relationships
www.melbourneinstitute.com
What’s In It? SCQ
 Health and well-being (SF36, Kessler 10, serious health conditions)
 Health behaviours (smoking, drinking, exercise, height / weight, diet)
 Social capital / relationships (satisfaction with family, social support, community
participation, religion)
 Neighbourhood characteristics
 Life events
 Time use
 Finances (stressful financial events, savings habits, risk preference, h’hold expend)
 Job attributes
 Parenting (parenting stress / work family gains and strains)
 Attitudes to work / gender roles / marriage
 Personality
www.melbourneinstitute.com
Indicators of Success
 We are still going!
– Funded renewed until wave 16
– And total funding has increased
 Response / attrition rates are good to
excellent
 Data usage is high
 Strong evidence of validity
www.melbourneinstitute.com
Response in 2001 was good
 HH response
– In-scope sample = 11,693
– 7682 cooperating households = 66% RR
 Individual response
– W1 individual sample = 15,127 persons
– 13,969 respondents = 92% RR
 Sample reasonably representative, but …
– Sydney residents under-represented
– People from a NESB under-represented
– Males less likely to complete a PQ
(but no less likely to be a CSM)
www.melbourneinstitute.com
Response in 2011 was better
80
70
60
50
40
30
20
10
0
HILDA Wave 1
(2001)
HILDA Top-up
(2011)
USoc: UKHLS
(2009-10)
SOEP H (2006)
www.melbourneinstitute.com
Response in 2011 was better
80
70
60
50
40
30
20
10
0
HILDA Wave 1
(2001)
HILDA Top-up
(2011)
USoc: UKHLS
(2009-10)
SOEP H (2006)
www.melbourneinstitute.com
Retention is High
(Annual Re-interview Rates: HILDA, BHPS & GSOEP)
100
95
90
%
BHPS*
85
GSOEP AB
80
75
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Wave
www.melbourneinstitute.com
Retention is High
(Annual Re-interview Rates: HILDA, BHPS & GSOEP)
100
95
90
%
HILDA
85
BHPS*
GSOEP AB
80
75
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Wave
www.melbourneinstitute.com
Fieldwork Outcomes: W1 Adults
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Dead
Overseas
Lost
NR - not issued
NR - non-contact
NR - contact
Respondent
W2 W3 W4 W5 W6 W7 W8 W9 W10
www.melbourneinstitute.com
Tracking Movers
22-23% of all hh’s change address b/w
each survey wave

Pre-field office activity
–
–
–
–





Notifications (1800#, change of address card, email)
Matching to Australia Post
Returns to sender
Move indicator variable
Other household members
Contact information collected at previous ivw
Neighbours
Other community resources
Online White Pages
www.melbourneinstitute.com
Minimising Refusals
MARKETING / RESPONDENT ENGAGEMENT
 PAL and brochure, newsletter / Stat report
 1800 number
PERSISTENCE
 2-3 stage fieldwork
 NRs re-issued in later waves
GOOD PEOPLE
 Selection and continuity of interviewers
 Training / interviewer engagement
RESPONDENT INCENTIVES
www.melbourneinstitute.com
Data User Numbers
Release
Total data orders
New users
Cumulative total
1
204
204
202
2
265
169
373
3
279
157
530
4
329
176
706
5
387
196
902
6
401
176
1078
7
455
199
1277
8
431
125
1402
9
500
141
1543
10 (@19 July)
426
132
1675
www.melbourneinstitute.com
Publication Count
Year
Journal
articles
Books /
book chapters
Other
publications
Working
papers
2002
5
0
0
3
2003
6
2
2
8
2004
24
4
8
15
2005
24
3
8
21
2006
25
1
19
23
2007
35
0
11
35
2008
38
0
23
35
2009
47
7
27
35
2010
52
6
20
30
2011
64
0
36
42
2012 / forthcoming
52
0
17
14
TOTAL
372
23
171
261
www.melbourneinstitute.com
Promoting Data Use











Well-documented, user-friendly data sets
User Manual
Other on-line tools (e.g., PanelWhiz)
Discussion Papers / Technical Papers
User training and panel data analysis courses
Biennial research conference
Membership of CNEF
Presentations to different stakeholders
Annual Statistical Report
Study-specific web site
User email list
www.melbourneinstitute.com
Research Uses: Key Features

Topic coverage extremely broad
www.melbourneinstitute.com
Research Topics: Just a Few Examples!
 Income and wealth
– Poverty dynamics
– Distribution of household wealth
– Retirement savings
 Labour supply / Unemployment
–
–
–
–
LFP and health
Family policy and couples LS
Impact of child care costs
Forgone earnings of mothers
 Employment
–
–
–
–
–
–
Working hours mismatch
Casual employment transitions
Part-time employment and wages
Job insecurity
Responses to long hours
Gender inequity
 Marriage and family
–
–
–
–
–
Patterns of cohabitation
Children’s living arrangements
Post-separation contact with children
Childlessness
Predictors of marital separation
 Subjective well-being
– Adaptation to life events
– Predictors / correlates of life
satisfaction
 Mental health and:
–
–
–
–
–
welfare reliance
retirement
housing affordability
joblessness
job quality
www.melbourneinstitute.com
Research Uses: Key Features


Topic coverage extremely broad
Three key types of studies
i. Innovative content / questions
ii. Unobserved heterogeneity
iii. Dynamics of change (and persistence)

Still many cross-sectional analyses
www.melbourneinstitute.com
Policy Impacts: Examples
 Key input into Government’s Pension Review
 Annual Wage Reviews
 RBA
– Household debt and risk
– Effect of the superannuation guarantee on
household saving
 Productivity Commission – Paid Parental
Leave report
www.melbourneinstitute.com
Keys to Success: Response





Expectations of fieldwork agency
Motivated interviewer workforce
Long fieldwork period
Persistence
Cash incentives
www.melbourneinstitute.com
Keys to Success: Other Ingredients






Champions (and lots of them)
Money (and lots of it)
Imitation
Good people
Many users
Luck
www.melbourneinstitute.com
Other Issues To Think About (I)
 Sample
– Population, dwellings, households
– Clustered / stratified
– Dealing with future immigration
 What mode?
– Interviewer administered vs self-administered
– Single mode vs mixed mode or multi-mode
 Respondent burden
 How to reach non-English speakers?
www.melbourneinstitute.com
Other Issues To Think About (II)
 Making use of technology
– Dependent data / On-line options
 How much value adding?
– Data cleaning / Weights / Imputation / Derived
variables
 Confidentiality vs data access
 Linkages to admin. data
 Scientific stewardship / Stakeholder
involvement
www.melbourneinstitute.com
The Development of a Successful
Household Panel Survey:
The HILDA Experience
Mark Wooden
Project Director, HILDA Survey