Cost-Efficient Framework for Data
Collection for CATI Surveys
Social Surveys Collection Research
Steering Committee (SSCRSC)
François Laflamme
December 2010
Statistics Canada • Statistique Canada
Past and current research
Paradata research has mainly directed its effort in
improving the current data collection process and
practices
Paradata research has also suggested that the
interviewer staffing levels are not always optimally
allocated with respect to the workload sample and
the expected productivity
2
Statistics Canada • Statistique Canada
2017-07-28
Challenges and objective
 One of the main challenges is to collect costeffective data while maintaining a high level of
quality
 According to the forecasted budget reductions, the
pressure has increased and will continue to grow
 The objective of the project is to propose a costefficient framework for data collection for CATI
surveys
3
Statistics Canada • Statistique Canada
2017-07-28
Cost-efficient framework
Five 5 dimensions:
1.
2.
3.
4.
5.
4
Metrics used for costing and budgeting surveys
Resources allocation within surveys
Resources allocation between surveys
Collection process and practices
Operational constraints
Statistics Canada • Statistique Canada
2017-07-28
1. Metrics used for costing and
budgeting surveys
Should be based on objective, measurable and
comprehensible assumptions
Should be monitored throughout the survey
collection
 Budget too high - low productivity
 Budget too low - more difficult to reach targets
Budget often based on previous survey cycles or
comparable surveys
 But it is exactly what we try to improve (e.g. low TPU)
5
Statistics Canada • Statistique Canada
2017-07-28
2 & 3. Resources allocation within
and between surveys
Within survey
 Interviewer staffing levels should consider sample
workload and collection progress in taking advantage of
the entire collection period
● Survey productivity decreases for all CATI surveys during
collection
Between surveys
 Many concurrent surveys compete for data collection
resources
6
Statistics Canada • Statistique Canada
2017-07-28
Resources allocation within and
between surveys
 What is the best strategy within survey?
Examples of Resources Allocation by Data Collection Days
Uniform
Proportional
Hours
Front Loading
Data Collection Days
7
Statistics Canada • Statistique Canada
2017-07-28
Resources allocation within and
between surveys
 The main idea is to try to maintain the survey productivity for
all surveys for a given RO (red line) between reasonable
boundaries around the average productivity
 How?
Daily Productivity, Some Social Surveys, January to March 2010
Winnipeg
100%
LFS
CCHS
All Surveys
80%
LFSTFC
SLID
Average All Surveys
60%
40%
20%
0%
30-Mar
27-Mar
24-Mar
21-Mar
18-Mar
15-Mar
12-Mar
09-Mar
06-Mar
03-Mar
Statistics Canada • Statistique Canada
28-Feb
25-Feb
22-Feb
19-Feb
16-Feb
13-Feb
10-Feb
07-Feb
04-Feb
01-Feb
29-Jan
26-Jan
23-Jan
20-Jan
17-Jan
14-Jan
11-Jan
08-Jan
05-Jan
02-Jan
8
Data Collection Day
2017-07-28
Resources allocation within and
between surveys
 The strategy is to distribute system time for a given period (day or
week) in function of the inverse of the proportion of the budget spent
(inversely correlated to productivity) to maintain productivity at RO level
Data Collection Day
Survey
CCHS
LFS
HES
CTUMS
…..
Survery X
1
3
4
5
6
7
8
9
10
11
12
13
14 … … 30
Budget CCHS
Budget LFS
Budget HES
Budget CTUMS
…..
Budget survey_X
C1
9
2
RO Budget (hrs)
C2
C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14
Ro Capacity - Daily Hours
Active surveys on that day
Past collection day
Next data collection day is the 3rd
Statistics Canada • Statistique Canada
C30
2017-07-28
Resources allocation within and
between surveys
In practice
 Need to get consistent budget figures for all surveys
 Need to get daily or weekly capacity system time
 Need to consider the active collection period of each
survey
 Need to take into account operational constraints
 Is it feasible? Can it be operationalized and automated?
The objective is to produce guidelines for RO to help
them allocate collection resources only at the
aggregate level.
10
Statistics Canada • Statistique Canada
2017-07-28
4. Collection process and practices
Focus of the past research
Some examples
 Best time to call
 Call scheduler features
● Cap on calls, customized time slices, Z-groups etc…
 Increased the proportion of evening shifts versus day
shifts to improve contact rate and productivity
 Active management
 Responsive collection design
No significant efficiency if budget and resources
allocation remain the same.
11
Statistics Canada • Statistique Canada
2017-07-28
5. Operational constraints
RO capacity
 Varies by day (interviewer availability)
 LFS week
 Holidays and special event
Average productivity
 Varies by RO and survey
Interviewer schedule in advance
Interviewer training across surveys
Etc…
12
Statistics Canada • Statistique Canada
2017-07-28
Comments
3 Separate projects
 Costing metrics
 Resources allocation within and between surveys
 Collection process and practices
 All need to take into account operational constraints
All dimensions have to be considered
simultaneously to succeed in improving the costefficiency of data collection.
13
Statistics Canada • Statistique Canada
2017-07-28
For more information, please contact
Pour plus d’information, veuillez contacter
François Laflamme
[email protected]
14
Statistics Canada • Statistique Canada
2017-07-28