Smarter task assignment or greater
effort: what makes a difference in
team performance?
Burgess, Propper, Ratto, Scholder, Tominey
Policy Studies Institute seminar, 26 June 2007
1
Introduction



Two main motivations:
Are financial incentives useful in the public sector?
Do they have a role to play in public service
reform?
Compared to theoretical work, relatively little
empirical evidence on incentives in organisations,
and how they work.
Policy Studies Institute seminar, 26 June 2007
2
What we do:



Study a pilot scheme of team-based financial
incentives in HM Customs & Excise (as was)
Use rich and disaggregate dataset based on their
personnel records and MIS
Research design allows diff-in-diff-in-diff, and
allows us to understand in detail how the scheme
worked.
Policy Studies Institute seminar, 26 June 2007
3
What we find:





At team level the incentive scheme did increase
performance
The incentive structure did raise individual
productivity.
Managers reallocated efficient workers to the
incentivised tasks; this happened disproportionately
in one of the treatment teams.
This reallocation was the more important
contributor to the overall outcome.
Why? Maybe career concerns …
Policy Studies Institute seminar, 26 June 2007
4
Plan




(Brief) details of the scheme
Data
Results:
 Team level
 Worker productivity
 Reallocation
 Decomposition
Conclusions – career concerns?
Policy Studies Institute seminar, 26 June 2007
5
UK experience on PRP in the public sector



November 1998 Public Services Productivity
Panel. Advise the government of ways of
improving the productivity and efficiency of
government departments.
Programme to improve efficiency and productivity
in the public sector. Makinson report (2000). Some
directions and particular emphasis on teamwork.
Team-based pilot schemes in DWP and HM
Customs and Excise.
Policy Studies Institute seminar, 26 June 2007
6
(Brief) details of the scheme 1





Pilot ran 1 April – 31 Dec 2002, with 12 trials
Team-based: team = division, a few offices. Bonus
for team as a whole.
Pairs of trial teams, plus blind control, with
different bonus schemes.
We use pair of VAT Assurance divisions, team 1
(154 workers); team 2 (158); and control team (281)
Mean potential bonus about 4% salary.
Policy Studies Institute seminar, 26 June 2007
7
Bonus amount
Jobband 2
Jobband 3
Jobband 4
Jobband 5
Jobband 6
Jobband 7
Jobband 8
Jobband 9
Jobband 11
Team 1
Team 2
420
515
550
453*
720
860
970
1145
1520
740
740
740
740
740
740
740
740
740
* only observation, however, the person did not work all 275 days, so did not get the
total bonus..
Policy Studies Institute seminar, 26 June 2007
8
(Brief) details of the scheme 2






Each incentive target was baseline + 5%
Aware of multi-task issues, and controlled.
Some tasks incentivised, some not.
Output split into different activities, different client
groups and different outputs
5 targets: 2 on visits (time), 3 on yield (££)
Note different production function effects of time
and yield.
Policy Studies Institute seminar, 26 June 2007
9
Activities of VAT
Trader groups
assurance office
Non-trader work
Other trader work
None-core
Unregisterable entity Trader audit work

divided into the following trader groups:
Non-VAT
Non-trader audit
New registration
Low risk
Medium risk
High risk
Exceptional risk
Large traders
Corporate groups
Branches
Insolvent
Deregistered
Missing traders
No. of audits
(time)


-
Policy Studies Institute seminar, 26 June 2007
Tax revenue
(yield)



-
10
Data
1 Apr 2001
1 Apr 2002
31 Dec 2001




31 Dec 2002
9 months of the scheme, same 9 months before
We have records for each worker, in each week, and
in each trader group.
Outcomes: time allocated by each worker to each
trader group (no direct measure of visits), total
yield per worker per trader group, positive and
negative yield per worker per trader group, and
productivity.
Focus on continuously employed, front-line staff
Policy Studies Institute seminar, 26 June 2007
11
Data 2


Time:
 Trader audit is more than half of time
 Mean time per worker/week/group is 8 hrs
 Mean groups worked on per w/w is 3
 Each incentivised group is about 30% of time
Yield:
 Mean +ve yield per w/w/g is £302k; but median is
just £5k  outliers! One obs w/w/g of £280m.
 Largest share of +ve yield from exceptional risk TG
 Av. –ve yield much smaller, follows same distribution
Policy Studies Institute seminar, 26 June 2007
12
Results




Team level
Worker productivity
Strategic task allocation
Decomposition
Policy Studies Institute seminar, 26 June 2007
13
Results - Team level




Exploit variation in incentivisation across:
 Time
 Treatment status
 Trader group
Diff-in-diff-in-diff, controlling for time effects,
team effects and TG effects.
Use median for anything involving ££
Short answer: scheme worked in T2 more than T1.
Policy Studies Institute seminar, 26 June 2007
14
Table 1a
For all trader
groups
Table 1b
Table 1c
For all incentivised
For all non-incentivised
Differences between incentivised
& pre-incentivised period
Differences between incentivised
& pre-incentivised period
Differences between incentivised
& pre-incentivised period
Team
1
Team 2
Variable
Difference:
Total Time
50.5
Total Yield 67193 73441.5 0
Positive
Yield
84983 80877
0
Negative
Yield
0
0
0
Productivity 137.54 120.13
Variable
Total Time
0
Team
1
Team 2
Variable
103.35 41.5
Table 1d
Difference-inDifference
Diff in diff:
C
Total Time
-3.8
T2-C
Variable
9
61.85
Total Time
0
0
0
89.78 9.17
Table 1e
Difference-inDifference
T1-C
Variable
73.89 -1.5
Total Yield 10901 31725
Positive
Yield
8370 46550.5
Negative
Yield
0
0
Productivity 58.70
C
Team Team
1
2
13.5
Total Yield
Positive
Yield
Negative
Yield
6406
0
0
6406
0
0
0
0
0
0
0
Productivity 58.24
T1-C
T2-C
Variable
T1-C T2-C
-2.3
75.40
Total Time
12.4
-14.5
6406
0
6406
0
0
0
Total Yield 67193 73441.5
Positive
Yield
84983 80877
Negative
Yield
0
0
Total Yield 10901 31725
Positive
Yield
8370 46550.5
Negative
Yield
0
0
Total Yield
Positive
Yield
Negative
Yield
Productivity 137.54 120.13
Productivity 49.53
Productivity 58.24
M*(T1-TC) – nonM*(T1-TC)
M*(T2-TC) – nonM*(T2-TC)
Total Time
-14.7
89.89
Total Yield
4495
31725
Positive Yield
1964
46550.5
Negative Yield
0
0
-8.71
80.61
Variable
Diff in diff in diff:
Productivity
-13.4 1.099
Table 1f
Difference-inDifference
80.61
Policy Studies Institute seminar, 26 June 2007
C
Total Time
0
15
Analysis – Worker level (unit: workertrader group)



Same five outcomes available at w/g level, adding
up over 9 months, and differencing
Regress:
yig  β.Xi  δ.Team i   .Incentiveig  γ.G g
Specifications:
 With & w/out non-VAT; with & w/out trimming
 Treated teams combined (intercept dummies),
and each treated team + control separately
Policy Studies Institute seminar, 26 June 2007
16
Specification 1
Both teams
Team 1
Team 2
Specification 2b
Both teams
Team 1
Team 2
Specification 3c
Both teams
Team 1
Team 2
Specification 4d
Both teams
Team 1
Team 2
Time
Yield
Positive yield
Negative yield
Productivity
16.662
(12.538)
-4.922
(12.373)
37.894 ***
(14.520)
862,226.531
(652,598.418)
136,162.869
(98,686.498)
978,548.109
(860,264.244)
879,553.131
(653,435.097)
145,919.115
(98,767.889)
1,001,806.908
(860,344.116)
-17,326.599
(12,221.249)
-9,756.246
(7,560.350)
-23,258.800
(21,407.965)
382.772
(3,608.746)
239.004
(470.589)
271.321
(408.845)
14.010
(12.902)
-4.581
(12.420)
37.937 ***
(14.541)
435,750.800
(412,809.891)
47,088.951
(42,239.992)
896,445.946
(860,973.361)
453,579.263
(414,757.543)
57,081.507
(43,773.840)
919,626.879
(860,979.605)
-17,828.463
(12,356.234)
-9,992.555
(7,575.689)
-23,180.933
(21,053.030)
40.633
(144.382)
166.134
(249.878)
58.324
(151.494)
24,517.457***
(5,292.030)
25,815.132***
(7,004.909)
29,781.494***
(6,927.112)
24,629.887***
(5,838.805)
25,696.292***
(7,864.822)
28,954.846***
(7,314.759)
-633.594***
(226.192)
-562.030**
(218.070)
-899.411***
(297.831)
28.494
(23.125)
70.981**
(32.555)
26.951
(22.341)
22,360.027***
(5,254.457)
25,180.406***
(6,873.621)
28,775.089***
(6,879.612)
22,154.517***
(5,842.350)
25,583.105***
(7,827.547)
28,803.199***
(7,288.997)
-585.759**
(236.258)
-549.367**
(219.232)
-879.165***
(296.545)
32.439
(23.020)
70.908**
(32.484)
27.019
(22.287)
a
* significant at 10%; ** significant at 5%; *** significant at 1% Significant coefficients (5% or better) are in bold.
Policy Studies Institute seminar, 26 June 2007
17
Worker Effort and yield
Specification 4
Team 1
Team 2
Yield
Positive yield
Negative yield
Productivity
25,180.406***
(6,873.621)
28,775.089***
(6,879.612)
25,583.105***
(7,827.547)
28,803.199***
(7,288.997)
-549.367**
(219.232)
-879.165***
(296.545)
70.908**
(32.484)
27.019
(22.287)
d
These are the coefficients on incentivisation
OLS on difference in outcome (so implicitly FE).
Unit is worker/group per 9-month spell, in £
This specification is trimmed of outliers (top & bottom 5%), and excludes
non-VAT outcomes.
Other controls: TG effects, team effects, years in grade, age, mean age
squared, gender, part-time/full-time worker, job band.
Policy Studies Institute seminar, 26 June 2007
18
Summary of results



At team level, treated teams spend more time on all trader
groups and recover more yield than the control team.
At individual level, the increase in productivity in
incentivised trader groups is larger in Team 1 than in Team
2, and statistically significant only for Team 1.
Mismatch between individual-level and team-level
outcomes:
 individual productivity increases more in T1 than in T2.
 at team level T2 performs better than T1
Different strategy at team level between the treated teams
Policy Studies Institute seminar, 26 June 2007
19
Results – Task Allocation



Identify efficient workers in pre-scheme period as
those in top quartile of productivity
Consider reallocation across a four way split:
incentivised vs non-incentivised tasks, efficient vs
non-efficient workers.
Is there a significant difference in the reallocation
of time on the incentivised and the nonincentivised trader groups by efficient workers and
other workers in each team?
Policy Studies Institute seminar, 26 June 2007
20
Non-inc
Horizontal axis = time on
incentivised trader groups
Increasing
total time
Inc
Vertical axis = time on nonincentivised trader groups
Teams can increase total
time on audit, and/or
reallocate time between the
two types of groups
Reallocating
fixed time
Time allocation
Policy Studies Institute seminar, 26 June 2007
21
Non-inc
Any point (x, y) derives from:
- an overall increase in time of (x – y)/2,
and
x, y
- a net reallocation of (x + y)/2.
Inc
- The latter is half of the vertical distance
between (x, y) and the positively sloped
45o line.
-The gross reallocation is twice that,
namely the full vertical distance between
(x, y) and the positively sloped 45o line
Time allocation
Policy Studies Institute seminar, 26 June 2007
22
Strategic task allocation: efficient workers
Change in time spent on NI_TG
T1
29
49
87
Change in time spent on I_TG
276
-89
C
T2
-122
Policy Studies Institute seminar, 26 June 2007
23
Individual level difference in time spent per month
Efficient workers
Others
Efficient workers
Others
All teams
Team 1
Team 2
Control team
131.94
56.89
t = 2.98
87.12
35.02
t = 1.50
275.91
98.84
t = 3.46
48.51
45.92
t = 0.06
Significant at 1%
Significant at 10%
Significant at 1%
Not significant
All teams
Team 1
Team 2
Control team
-63.56
26.04
t = -4.30
29.31
54.05
t = -0.741
-122.17
-8.31
t = -2.66
-89.29
28.73
t = -3.75
Significant at 1%
Not significant
Significant at 1%
Significant at 1%
Incentivised
trader groups
Nonincentivised
trader groups
Reallocation between incentivised and non-incentivised trader groups
Efficient workers
Others
Team X
Team Y
Team X
Team Y
All teams
Team 1
Team 2
Control team
195.50
30.84
t = 4.56
57.80
-19.03
t = 1.52
398.08
107.15
t = 3.91
137.79
17.18
t = 2.1386
Significant at 1%
Significant at 10%
Significant at 1%
Significant at 5%
Team 1 vs. Team 2
Team 1 vs. Team C
Team 2 vs. Team C
57.80
398.08
t = -4.25
57.80
137.79
t = -1.13
398.08
137.79
t = 2.92
Significant at 1%
Not significant
Significant at 1%
Team 1 vs. Team 2
Team 1 vs. Team C
Team 2 vs. Team C
-19.03
107.15
t = -2.96
-19.03
17.18
t = -1.0241
107.15
17.18
t = 2.21
Significant at 1%
Not significant
Significant at 5%
Policy Studies Institute seminar, 26 June 2007
Efficient
workers
Reallocation of
efficient workers
Reallocation of
other workers
24
Time change per trader group
Efficient worker * New registrations
Efficient worker * Low risk
Efficient worker * Median risk
Efficient worker * High risk
Efficient worker * Exceptional risk
Efficient worker * (999) Large traders
Efficient worker * Corporate groups
Efficient worker * Branches
Efficient worker * Insolvent
Efficient worker * Deregistered
Efficient worker * Missing traders
Team 1
Team 2
-15.001
(21.992)
-3.071
(8.022)
14.049
(9.806)
-15.169
(20.790)
62.801 **
(25.398)
-14.579
(23.095)
0
(0.000)
0.495
(4.063)
9.547 **
(3.865)
3.002
(3.116)
10.339
(6.056)
43.716 *
(25.351)
-26.502 ***
(9.769)
-18.213 **
(8.882)
-45.998
(29.948)
122.785 ***
(42.559)
-55.919
(41.144)
-9.942
(6.982)
0
(0.000)
4.715
(3.299)
9.241 **
(3.895)
-0.232
(2.515)
Policy Studies Institute seminar, 26 June 2007
25
Results - Decomposition



How much of the change in a team’s yield is due to
increased worker effort, and how much to better
deployment of workers?
Tijt is time spent by i on group j period t, and yijt is
the corresponding productivity
Decomposition of total team yield:
Y  Y2  Y1  
i
 T
ij 2
 Tij1  ij1  
j
Policy Studies Institute seminar, 26 June 2007
i
 
ij 2
 ij1  Tij 2
j
26
Decomposition of the change in total yield
Change in yield due to:
Reallocation
Increased Effort
Total
Reallocation as %
Increased Effort as %
Team 1
Team 2
Control Team
10,500,514
3,445,083
13,945,597
19,785,364
1,378,579
21,163,943
4,330,667
752,514
5,083,181
75.3 %
24.7 %
93.5 %
6.5 %
85.2 %
14.8 %
We have taken out the highest single yield datapoint in T2 of £280m
Policy Studies Institute seminar, 26 June 2007
27
Conclusions


The incentive scheme:
 raised individual yield and productivity
 led to the reallocation of efficient workers towards
the incentivised tasks.
 Reallocation was the more important contributor.
Comparing the teams:
 The increase in individual yield collected was
essentially the same in the two teams.
 But T2 engaged in reallocation to a significantly
greater degree than T1
 T2 hit its targets and collected the bonus.
 Why?
Policy Studies Institute seminar, 26 June 2007
28


Conclusions 2
Why:
 Staff composition: more experience in T2? No.
Different managerial strategy prior the scheme? No.
 Nature of bonus scheme? No.
Intrinsic incentives – career concerns?
 Office managers in T2 were younger, and had been
in that job band for fewer years.
 Division manager in T2 was much younger than in
either the control or T1, and had less experience.
 Managers in T2 may have been more driven to
perform well for longer term career concerns
Policy Studies Institute seminar, 26 June 2007
29
Policy Studies Institute seminar, 26 June 2007
30
Managers Profile
Job band 9
All
Team 1
Team 2
Control
gender
0.24
0.10
0.33
0.27
age
50.62
52.90
46.78
51.40
job_band
9.00
9.00
9.00
9.00
jobband_yrs
4.09
4.60
2.78
4.53
service
27.94
27.70
25.44
29.60
pay
30970.32 30829.90 29558.22 31911.20
gmpay
2526.78 2485.22 2449.04 2601.13
gmpot
2526.78 2485.22 2449.04 2601.13
otm
0.00
0.00
0.00
0.00
pt
0.12
0.10
0.11
0.13
ptpw
28.33
24.00
.
30.50
sick
0.03
0.00
0.11
0.00
days
271.61
268.90
275.00
.
bonus
915.95
1079.40
734.33
.
Job band 11
All
Team 1
Team 2
Control
0.67
1.00
1.00
0.00
48.00
48.00
40.00
56.00
11.00
11.00
11.00
11.00
4.33
5.00
2.00
6.00
28.00
27.00
21.00
36.00
39651.00 38667.00 38279.00 42007.00
3304.25 3222.25 3189.92 3500.58
3304.25 3222.25 3189.92 3500.58
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
.
.
.
.
0.00
0.00
0.00
0.00
275.00
275.00
275.00
.
1130.00 1520.00
740.00
.
Policy Studies Institute seminar, 26 June 2007
31
The Teams
There are 12 teams, consisting of trial sites in RBS and LBG. These are:
-
VAT Assurance Division, Solent, RBS South
-
VAT Assurance Division, North East, RBS Central
-
Excise Assurance, RBS Scotland
-
Excise Assurance, RBS North
-
International Trade Assurance, RBS North
-
International Trade SME Assurance, RBS Scotland
-
Risk Team (Operational Analysis Division), RBS North
-
Risk Team, RBS Northern Ireland
-
National Function – Registration Unit
-
National Function – Debt Management Unit
-
National Function - Nation Advice Service
-
LBG Scotland and Northern Ireland
Policy Studies Institute seminar, 26 June 2007
32
Bonus award process
Are the obligations/targets in
all other areas of the team
activity which are not subject
of the incentives targets met?
No
Yes
The team qualifies for admission to bonus scheme
Does the team meet the
baseline target and at least
50% of the stretch on all
incentive targets?
No
Yes
Payment of bonus:
- 100% bonus if all the incentive targets are met in full
- 50% bonus if incentive targets are not met in full but
baseline target and at least 50% of the stretch is met
on all incentive targets
Policy Studies Institute seminar, 26 June 2007
33
Team staff mean characteristics.
All staff
Continuous frontline staff
Overall
Team 1
Team 2
Control
Overall
Team 1
Team 2
Control
Gender
0.47
0.39
0.44
0.52
0.44
0.40
0.40
0.50
Age
44.2
45.7
45.4
42.8
44.6
45.0
45.5
43.7
Job band
5.9
6.2
5.9
5.8
6.2
6.3
6.3
6.0
Years in job band
3.9
4.0
4.3
3.6
4.2
4.0
4.7
4.1
Years at Customs &Excise
18.1
19.1
19.1
17.1
19.5
19.3
20.0
19.2
Annual pay (£)
20247
20797
20066
20047
20671
20742
20789
20551
Gross monthly pay (£)
1594
1630
1599
1571
1632
1619
1657
1625
Gross monthly pay + overtime (£)
1599
1632
1599
1582
1639
1621
1657
1639
Overtime per month (hours)
0.38
0.16
0.02
0.70
0.44
0.15
0.02
0.88
Part-time marker
0.16
0.16
0.13
0.17
0.16
0.19
0.12
0.16
Part-time hours per week
24.8
24.1
-
25.1
24.7
23.8
-
25.2
Days of sickness
1.1
1.1
1.5
0.9
1.0
1.2
1.3
0.8
Days in scheme
260.2
258.8
261.5
-
264.0
262.4
265.6
-
-
688
681
-
-
695
696
-
Potential bonus (£ per person)
Policy Studies Institute seminar, 26 June 2007
34
Time series variation in time.
4000
6000
8000
10000 12000 14000
Total monthly time (hours)
2001m1
2001m7
2002m1
Month
Trial Team 1
Control Team
2002m7
2003m1
Trial Team 2
Policy Studies Institute seminar, 26 June 2007
35