What do we know about mixed-device
online surveys and mobile device use in
the UK?
Olga Maslovskaya, Gabriele Durrant, Peter WF Smith
14 March, 2017, NTTS conference, Brussels
Motivation
• Internet used daily by 82% of adults, smartphones are owned
by 71% of adults in the UK (ONS 2016)
• Mixed-mode designs for cost savings (e.g., Understanding
Society)
• Surveys moving to online designs with different devices as
options for respondents
• UK 2021 Census – plan to collect 75% of household responses
through online data collection
• Can no longer be expected that all participants would use PCs
or laptops in online surveys
• No research on characteristics of people in mixed-device
online surveys in the UK
Research Question
• What are the characteristics of people choosing to
use different devices in online survey completion in
the UK?
Literature
• Netherlands: de Bruijne and Wijnant 2014,
Toepoel and Lugtig 2014
• Germany: Bosnjak et al. 2013
• US: Peterson 2012
• Spain, Portugal, Argentina, Brazil, Chili,
Colombia and Mexico: Revilla et al. 2016
• UK: no studies yet
Previous Findings
Toepoel &
Lugtig ‘14
de Bruijne & Bosnjak et
Wijnant ‘14
al. ‘13
Revilla et
al. ‘16
Age
Y
Y
Y
Y: B, Ch, C, S Y
Gender
N
Y
N
Y: S, P
Education
N
Y
N
N
Ethnicity
Peterson
‘12
Y
Y
Employme
nt status
Y
HH size/
composition
Y
Children in
HH
Y
HH income
Y
Urban/rural
N
Y
Y: (P, A)
N
Data
• Innovation Panel of Understanding Society Waves 7-8 (IP7 and IP8)
• Community Life Survey (CLS) 2014-2015
• European Social Survey (ESS) – Round 6 – not publicly available but
descriptive analysis conducted by the ESS
• 1958 National Child Development Survey (NCDS)
• Second Longitudinal Survey of Young People in England (LSYPE2)
Wave 4 – not publically available yet but descriptive analysis was
conducted by Kantar Public
Methods
• Descriptive analysis (proportions and Chi-square
tests)
• Binary Logistic Regression
• Multinomial Logistic Regression
Device Used Variable 1
Main variable 1
Survey
PC/laptop
Mobile devices
IP7
621 (81.6%)
140 (18.4%)
IP8
2030 (90.3%)
217 (9.7%)
ESS6
540 (91.7%)
49 (8.3%)
CLS 14-15
1606 (72.5%)
609 (27.5%)
NCDS
5056 (86.5%)
790 (13.5%)
LSYPE2
1737 (60.6%)
1128 (39.4%)
Device Used Variable 2
Main variable 2
Survey
PC/laptop
Tablet
Mobile phone
2030 (90.3%)
184 (8.2%)
33 (1.5%)
1606 (72.5%)
567 (25.6%)
42 (1.9%)
1737 (60.6%)
485 (16.9%)
643 (22.4%)
IP7
IP8
ESS6
CLS 14-15
NCDS
LSYPE2
Results: Descriptive Analysis - Availability and Significance of
Variables
IP7
IP8
ESS
CLS
NCDS
LSYP2
Age
Y*
Y*
Y
Y*
N/A
N/A
Pension age
Y*
Y*
Gender
Y*
Y*
Y
Y*
Y*
Y*
Marital status
Y*
Y*
Y
Y*
Y
N/A
Ethnicity*
Y
Y
Y
Y
Y
Y*
Y
Y
Y
Y*
Y
Y*
Religion
Education
Y
Employment status Y
Accommodation
Tenure
N/A
Y
Y
Y*
Y
Y*
Y*
Results: Descriptive Analysis 2 - Availability and Significance of
Variables
IP7
IP8
ESS
CLS
NCDS
LSYPE2
HH income
Y
Y*
Y
Y*
Y
N/A
Number of cars
Y
Y*
Y*
N/A
Urban/ rural
Y
Y
Y
GOR
Y
Y
Y
Country of residence
Y
Y
Y
HH size
Y
Y
Y
Y*
Children in HH
Y*
Y*
Y
Y*
Y
N/A
General health
Y
Y
Y
Internet use
Y
Frequency of
Internet use
Y*
Y*
Y*
Summary of Descriptive Results
• Younger – mobile devices, older – PCs and laptops – consistent with
all other studies
• Female – mobile devices – consistent with other studies
• Employed – mobile devices, unemployed – PCs and laptops –
consistent with other studies
• Higher income – mobile phones – consistent with other studies
• Larger households – mobile devices – consistent with Revilla et al.
(2016) but not with Toepoel and Lugtig (2014)
• Children in household – mobile devices – consistent with other
studies
• Education not significant – consistent with other studies
Modelling Results IP7 (N=695): Binary Logistic Regression
(tablet=1)
Variable
Category
Gender
Male
Female
Marital Status
Tenure
Coefficient (SE)
0.540 (0.207)**
Single
Married
-0.078 (0.238)
Divorced or separated
-0.188 (0.376)
Widowed
1.367 (0.461)**
Owned outright or with
mortgage
Rented from local
authority or housing
association
0.423 (0.325)
Rented privately
0.841 (0.339)*
Modelling Results NCDS (N=5,844): Binary Logistic Regression predicted probabilities of using tablets
Significant effects:
1. Gender
2. Frequency of Internet use
3. Gender*Frequency of
Internet use
Modelling Results IP8 (N=2,214): Multinomial Logistic Regression
Variable
Category
Tablet
Phone
PC/laptop
Phone
Gender
Male
0.500
(0.163)**
0.549
(0.395)
-0.500
(0.163) **
0.049
(0.422)
0.355
(0.177)*
0.820
(0.344)
-0.355
(0.177)*
0.465
(0.466)
1st quartile
-0.692
(0.236)**
0.003
(0.698)
0.692
(0.236)**
0.694
(0.730)
2nd quartile
-0.594
(0.220)**
0.817
(0.561)
0.594
(0.220)**
1.412
(0.595)*
3rd quartile
-0.557
(0.209)**
0.679
(0.553)
0.557
(0.209)**
1.235
(0.584)*
Female
Employment
Unemployed
Employed
HH income
4th quartile
Modelling Results CLS (N=2,125): Multinomial Logistic
Regression
Variable
Category
Gender
Male
Female
Employme
nt
Phone
PC/laptop
Phone
0.539
0.031
(0.106)*** (0.332)
-0.539
(0.106)***
-0.508
(0.340)
0.378
(0.136)**
-0.378
(0.136)**
0.202
(0.446)
0.846
(0.130)***
-0.035
(0.380)
Unemployed
Employed
Children in
HH
Tablet
0.580
(0.435)
Yes
No
-0.846
-0.882
(0.130)*** (0.371)*
Modelling Results CLS – continued
Variable
Category
Tablet
Phone
PC/laptop
Phone
Age
70+
16-19
-0.278
(0.320)
1.384
(1.270)
0.278
(0.320)
1.662
(1.292)
20-29
-0.424
(0.249)
2.266
(1.096)*
0.424
(0.249)
2.691
(1.113)*
30-39
0.048
(0.243)
1.553
(1.150)
-0.048
(0.243)
1.505
(1.164)
40-49
0.142
(0.235)
0.412
(1.215)
-0.142
(0.235)
0.270
(1.227)
50-59
-0.008
(0.224)
0.714
(1.178)
0.008
(0.224)
0.722
(1.190)
60-69
0.155
(0.200)
0.659
(1.169)
-0.155
(0.200)
0.503
(1.179)
Conclusions
• This is the first study to look at characteristics of respondents in
different device group in mixed-device online surveys in the UK
• Significant variables in different surveys: age of respondent, gender,
marital status, employment status, religion, household
composition/size, children in household, household income, number
of cars, and frequency of Internet use
• Results are useful for targeting of certain groups more efficiently for
survey participation, might help increase response rates and reduce
costs
• Results are instrumental in better understanding of trends in
different device use in preparation for the UK 2021 Census
Limitations
• Sample sizes are too small for mobile device use in
some surveys (respondents were discouraged to use
mobile devices or mobile devices were blocked)
• Data is not publically available for the analysis for
some surveys
Future Work
• Would be useful to repeat analysis on larger datasets
which will soon become available (e.g.,
Understanding Society Wave 8)
• Data quality issues by mode and by device used
within online mode of data collection should be
addressed