Summary
The Understanding Society Innovation Panel invites proposals for the content and design of its 2018
survey. Proposals must have an experimental or methodological element, and should make use of
the longitudinal nature of the survey. Proposals will be evaluated on the basis of their scientific
quality and publication potential. There are no costs to successful applicants, unless the proposal
includes non-standard survey elements. Deadline for submission is 30 March 2017.
Released 6 February 2017
This is an invitation to academics and researchers to submit proposals for studies to be incorporated
on the Understanding Society Innovation Panel wave 11. Data collection will take place in spring
2018.
The deadline for proposals is 30 March 2017.
The Innovation Panel is a unique resource for innovative longitudinal experimental and
methodological research with international impact. Its purpose is to develop key innovations in
survey methods and content that will ensure the future success of the Understanding Society survey,
to generate important new knowledge disseminated through publications in leading international
journals, and to provide a base for associated studies using new and innovative mixed method
approaches.1
New experimentation and methodological tests
This call is for proposals for experiments and methodological studies. Proposals may address
substantive social science questions, or survey methodological questions, or both. Methodological
studies may relate to the design of survey instruments (e.g. question wording, item order, etc.) or to
features of the survey design (e.g. procedures intended to reduce non-response or to improve
fieldwork efficiency).
Proposals must meet the following criteria:
1. The research design must have an experimental or methodological element. Proposals for new
content will only be considered if they include a research design for evaluating the proposed
questions.
1
See the separate call for Associate Studies:
https://www.understandingsociety.ac.uk/research/get-involved/associated-studies
1
2. The research design should have a longitudinal element. Proposals that could be implemented
on a cross-sectional are unlikely to be considered. This could be a design that is
• conditional on past characteristics or dynamics measured in previous interviews, or
• which requires measurement of outcomes later in time.
3. The expected sample sizes must be sufficient to answer the proposed research questions.
4. There should be no conflict with core Understanding Society methods tests or with requirements
for experiments accepted in previous waves (see Annex 1).
5. The study should not pose a threat to the future of the panel.
6. The development and implementation must be within the resources of the Innovation Panel, in
terms of systems costs, development time and questionnaire time. Bear in mind that the
Innovation Panel questionnaire is only 40 minutes on average, including a self-completion
element.
7. The proposal must include a draft specification of all proposed questions and other required
text.
8. The proposal must be feasible in terms of complexity for development and implementation.
Proposals that meet the criteria above are evaluated for their scientific merit and
value for money:
9. The quality and publication potential of the research design, including
• innovative research questions or design,
• preferably on topics for which there is little or no existing evidence,
• careful research design, including the specification of the study design and implementation,
consideration of sample sizes, and adequate analysis plan.
10. The research questions should be within the general remit of Understanding Society.
Priority will be given to experiments related to health, health behaviours, and wellbeing or the use
of new technologies. In addition, proposals relating to Understanding Society core topic areas are
preferred:
•
•
•
•
Education
Employment
Family and household
Income, housing, wealth, expenditure & deprivation
Proposers should note wave 11 will involve experimental mixed mode data collection. The IP10
refreshment sample will be issued to face-to-face interviewers. Of the original and previous
refreshment samples 2/3 of households will be invited to complete the survey online and nonrespondents will be followed up by face-to-face interviewers. The remaining 1/3 of households will
be issued to face-to-face. In all samples there will be final follow-ups of non-respondents by
telephone. This will constrain other methodological studies. The survey instruments will contain the
same items in each mode, with only minimal differences.
It should also be noted that data collected on the Innovation Panel include interviewer observation
items, call records and time stamp data for questionnaire sections.
2
Standard data collection costs will be borne by Understanding Society (but see point 6 above) and
there will be no cost to successful proposers. Costs for non-standard elements of data collection
(e.g. task-related incentive payouts or additional mailings) will be borne by proposers.
Background: Understanding Society
Understanding Society: the UK Household Longitudinal Study is a major research study designed to
enhance understanding of life in the UK and how it is changing. The study, funded primarily by the
UK Economic and Social Research Council, takes a sample of 40,000 households containing around
100,000 individuals and attempts to interview all household members annually. A large boost
sample of ethnic minority persons is also included. The first wave began in January 2009 with
interviewing
spread
over
24
months.
More
information
is
available
at
https://www.understandingsociety.ac.uk/
Background: The Innovation Panel
The Innovation Panel is an important and integral part of the design of Understanding Society. It
consists of the original sample of around 2,500 persons, clustered within households (for more
details, see Lynn 2009), plus refreshment samples of around 700 persons added in 2011, 2014 and
2017. Interviews have been attempted with all adult sample members at annual intervals starting
with wave 1 in 2008. Wave 10 will shortly be going into the field (spring 2017). Subsequent waves of
interviewing will continue to be carried out at annual intervals (subject to funding), replicating the
household panel survey design employed by Understanding Society, whereby attempts are made to
re-interview all sample members regardless of changes in household composition or geographical
location, as well as any other (non-sample) members of the current households of sample members.
To date a range of experiments and tests have been incorporated into the Innovation Panel. For a
list of experiments carried in waves 1-10 see the Annex 1; for initial findings from waves 1-8 see
Burton et al 2008, 2010, 2011, 2012, 2013 and Al Baghal et al 2014, 2015, 2016.
The questionnaires used at previous waves of the Innovation panel are available at
https://www.understandingsociety.ac.uk/documentation/innovation-panel/questionnaires
Submitting a Proposal
Proposals should be submitted by 30th March 2017 using the online application form available at
https://www.understandingsociety.ac.uk/research/get-involved/innovation-panel-competition.
In preparing proposals, applicants are encouraged to consult with Annette Jäckle
([email protected]) about how their study could be designed to meet the Innovation Panel criteria.
Please send a brief description of your study and how it would meet criteria 1 and 2 above.
Using the online application form, please describe clearly the following aspects of your study:
•
The objectives of the research: the research questions and motivations, why the research is
innovative and important, its value in a longitudinal context, why previous research is
insufficiently informative, and justification over potential alternative designs.
3
•
•
•
•
•
The experimental or methodological design of the study: the experimental manipulations
and randomisation method, proposed questions, and, if your proposal does not include an
experiment, your research design for evaluating the proposed questions. Please include
question wordings or other required text as an appendix.
The longitudinal element of the research: why a longitudinal survey is necessary for this
study, whether the study would use information collected in the past, or other information
that would need to be collected in a future wave. Note that, unlike in previous years, we will
not be able to repeat experiments in the following wave.
Sample required: whether the experiment/test would be carried out on the whole sample,
or only certain sub-samples. Please pay attention to the sample size you will require for
analysis. Please provide power calculations where appropriate. Note that the number of
respondent households is expected to be roughly 1,300, and the number of respondent
adults giving full interviews roughly 2,100.
Resource implications: expected required questionnaire time, justification for estimated
questionnaire time (e.g. number of additional questions, whether all sample members
asked, etc.), additional questions from the Innovation Panel questionnaire which you would
need in the same interview as your study, additional cost elements (e.g. for printing,
mailings, incentives, capital equipment) and the source of the budget for these.
Planned research outputs: how you plan to analyse the results of the experiment or how
you will evaluate the proposed questions if your design does not include an experiment, and
your plans for publishing the results.
Proposals must be accompanied by a CV (maximum 2 pages) for each named proposer, the draft
specification of questions or other required text, and a list of references cited in the Case for
Support.
All proposals will be screened as to whether they meet the necessary criteria outlined above.
Proposals that do meet the criteria will be reviewed for feasibility and their scientific merit will be
evaluated by a review panel comprising members of the Understanding Society Principal
Investigator’s team and Methodological Advisory Committee.
All proposers will receive preliminary notification of the review panel’s decisions in July 2017. Final
acceptance will be conditional on fully establishing the feasibility of the proposed study. Successful
proposers will be expected to work with the Innovation Panel survey team to develop and finalise
the details of the implementation of the experiment. This will include checking the questionnaire
specification or other relevant documents, and testing the computerized questionnaire script. Once
data are available, proposers will be expected to analyse and report the main outcome(s) in a
summary form appropriate for inclusion in a Working Paper. Successful proposers will be asked to
sign a memorandum of agreement indicating their willingness to carry out these development and
analysis activities.
Proposers are expected to publish their research based on the resultant data. The data will be made
available to proposers as soon as possible and in advance of general release via the UK Data Archive.
4
References
Al Baghal, T. (ed.) 2016. "Understanding Society Innovation Panel Wave 8: Results from
Methodological Experiments." Understanding Society Working Paper 2016-02.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2016-02
Al Baghal, T. (ed.) (2015) Understanding Society Innovation Panel Wave 7: results from
methodological experiments, Understanding Society Working Paper 2015-03.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2015-03
Al Baghal, T. (ed.) (2014) Understanding Society Innovation Panel Wave 6: Results from
Methodological Experiments, Understanding Society Working Paper 2014-04.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2014-04
Burton, J. (ed.) (2013) Understanding Society Innovation Panel Wave 5: Results from Methodological
Experiments, Understanding Society Working Paper 2013-06.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2013-06
Burton, J. (ed.) (2012) Understanding Society Innovation Panel Wave 4: Results from Methodological
Experiments, Understanding Society Working Paper 2012-06.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2012-06
Burton, J. (ed.) (2011) Understanding Society Innovation Panel Wave 3: Results from methodological
experiments. Understanding Society Working Paper 2011-05.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2011-05
Burton J, Laurie H & Uhrig S C N (ed.s) (2010) Understanding Society innovation panel wave 2:
Results from methodological experiments. Understanding Society Working Paper 2010-04.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2010-04
Burton J, Laurie H & Uhrig S C N (ed.s) (2008) Understanding Society: Some preliminary results from
the wave 1 innovation panel. Understanding Society Working Paper 2008-03.
https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2008-03
Lynn P (2009) Sample design for Understanding Society. Understanding Society Working Paper 200901. https://www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2009-01
5
Further Information
Innovation Panel questionnaires:
https://www.understandingsociety.ac.uk/documentation/innovation-panel/questionnaires
User Guide for Innovation Panel waves 1 to 8:
https://www.understandingsociety.ac.uk/documentation/innovation-panel
6
Annex 1: Experiments and methodological tests carried in previous waves of the Innovation Panel
Topic
Type
Wave
Respondent incentives to increase survey participation
Subsetting content: effects of previous exposure to survey questions on responses (panel conditioning)
Measurement of household consumption
Measurement of labour market status with and without showcards
Measurement of job satisfaction using 7 versus 11 point rating scales
Measurement of unearned income with and without showcards
Measurement of personal consumption
Procedures
Q design
Q design
Q design
Q design
Q design
Q design
1, 2, 3, 4, 5, 6,7
1, 2, 3, 4, 5, 6
1
1
1
1
1
Testing mixed telephone and face-to-face interviewing and transfer protocols between modes
Effects of advance letters versus cards on response
Effects of showcards versus no showcards on measurement
Measurement of job satisfaction: interviewer versus self-completion, full versus end labelling of scales points,
showcards versus no showcards
Measurement of life satisfaction: interviewer versus self-completion, full versus end labelling of scales points,
showcards versus no showcards, position in questionnaire
Measurement of identity
Measurement of change: explicit versus implicit requests for dates of events, effects of ambiguous question wording
Improving measurement of wealth
Measurement of change: context effects, wording of dependent interviewing questions
Branched versus unbranched rating scales for measuring attitudes
Procedures
Procedures
Procedures
Q design
2
2
2
2, 3, 6
Q design
2, 3, 6
Q design
Q design
Q design
Q design
Q design
2
2, 3, 4
3
3, 4
3, 4, 5
Paper versus computer assisted collection of self-completion questionnaire
Inter-wave mailing experiment testing different ways of asking respondents to register on the participants website
Improving measurement of interviewer observations about the sample address
Effect of content of re-issue letter on response
Early bird: methods of encouraging respondents to phone interviewer ahead of fieldwork to schedule appointment
Context of questions asking for consent to link to benefit records
Measuring respondent preferences about modes of data collection
Context effects on fertility decisions
Procedure
Procedure
Procedure
Procedure
Procedure
Q design
Q design
Q design
4, 5, 6
4
4
4
4
4
4, 5, 6
4, 5
7
Topic
Type
Wave
Using a mix of face-to-face and web data collection
Maximising response to web surveys: incentives / on which day of the week to send email invitations
Manipulating advance letter text to increase response
Wording effects of dependent interviewing questions on the amount of change measured
Measurement of life satisfaction
Response to “smiley faces” versus text-based scales in child self-completion instruments
Assessing the feasibility of directly measuring household energy use
Measuring partner satisfaction with work division using vignettes
Subjective expectations about the returns to schooling and the decision to go to university
Questions about twins
Measuring change in self-assessed disability
Testing quick expenditure questions
Data quality when switching from face to face to web mode in a panel survey
Feasibility of measuring prenatal testosterone exposure through finger length ratios in face-to-face and web
interviews of the general population
Procedures
Procedures
Procedures
Q design
Q design
Q design
Q design
Q design
Q design
Q design
Q design
Q design
Q design
Q design
5, 6
5
5
5, 7
5
5, 6
5, 6, 7
5, 6
5
5
6, 7
6
6
6, 7
Targeted advance letters
Risk and time preferences (Associated Study)
Procedures
Q design
6
6, 7
Time Use Diary (Associated Study)
Separating systematic measurement error components using MTMM in longitudinal studies
Experiments for Survey Question Evaluation in Multiple-Country Contexts
How do people think about environmental taxes? A survey experiment
Impact of response scale direction on survey responses
Social desirability in general health questionnaire (GHQ-12)
Enhancing respondent motivation through asking tailored interesting questions
Keeping in contact: The effect of multiple contacts
The use of conditional and unconditional incentives
Examining the Validity of Interviewers' Ratings of Respondents' Health
Masking opposition to immigration: an longitudinal approach to measure social desirability bias
Q design
Q design
Q design
Q design
Q design
Q design
Q design
Procedures
Procedures
Q design
Q design
7
7, 8, 9
7, 8
7
7, 8
7
7
7
7
8
8,9
8
Topic
Type
Wave
A comparison of self-reported sexual identity using direct and indirect questioning
Subjective expectations about the returns to schooling and the decision to go to university
Format of health satisfaction questions
What do the general population regard as “successful ageing”?
Measuring household finances: individual Income summary screen and Benefit Unit Finance module
Targeted weekday invitation emails for web survey
Improving the representation of separated parents, particularly non-resident parents, in the UKHLS
Linking social media data and social survey data on a large-scale longitudinal survey
Financial management and perceptions of ownership of money within couples
Improving consent to link to the electoral register, and assessing the impact of Individual Electoral Registration on
under-registered social groups
Applying prospect theory to the request for repeated participation in a longitudinal survey
Q design
Q design
Q design
Vignettes
Q design
Procedures
Q design
Linkage
Q design
Q design
8,9
8,9
9
9
9
9
10
10
10
10
Procedures
10
9