Smarter task assignment or greater effort: what makes a difference
in team performance?
Simon Burgess
University of Bristol, CMPO and CEPR
Carol Propper
University of Bristol, CMPO and CEPR
Marisa Ratto
PSI
Stephanie von Hinke Kessler Scholder
CMPO
Emma Tominey
University College London
March 2007
Abstract
This paper exploits a rich and disaggregate dataset to examine the impact of financial incentives in
a large organisation. The organisation is responsible for the collection of all indirect tax in the UK.
It undertook a randomised control trial of use of incentives in a team based scheme. We use data
from the personnel records and the management information system of the organisation to examine
the impact of the scheme. We first examine whether, at team level, the incentive scheme had an
impact on performance against the targets set in the scheme and then two potential sources of any
impact: increased effort from individuals versus reallocation of manpower by team managers. We
find that at the team level the incentive scheme did increase performance. We find that the
incentive structure also raised individual performance. We identify efficient workers and
investigate the extent to which managers reallocate these workers to the incentivised tasks once the
scheme began. We show that this happened, and happened disproportionately in one of the two
treatment teams. We show that this reallocation was the more important contributor to the overall
outcome.
Keywords: Incentives, Public Sector, Teams, Performance.
Acknowledgements: This work was funded by the Department for Work and Pensions (DWP), the Public Sector
Productivity Panel, the Evidence-based Policy Fund and the Leverhulme Trust through CMPO. The views in the paper
do not necessarily reflect those of these organisations. Thanks to individuals in the HM Custom and Excise for helping
to secure the data for us, particularly Sue Macpherson and Pat Marsh. Thanks for comments to seminar participants at
Bristol.
Corresponding author: Simon Burgess. [email protected]
1. Introduction
The evidence on the impact of financial incentives on the behaviour of individuals in organisations
is relatively limited (Dixit 2002). Explicit incentives have been advocated for use in both the
private and public sectors. Theorists have addressed the role of such incentives in the public sector,
drawing attention to a set of features such as multiple tasks, multiple principals and missions, all of
which suggest that even if there is no difference in the inputs of the production process between the
private and public sector, there are some specific features related to outputs and to the way public
sector agencies are structured which mean that incentives might be expected to have different
consequences in public organisations (e.g Dixit 2002, Prendergast 1999, Baker 2002, Francois
2000, Besley and Ghatak 2005). However, the empirical evidence on the use of incentives in the
public sector is quite scant, a number of surveys noting that the advance of theory has outstripped
the available evidence (Dixit 2002, Burgess and Ratto 2003).
This paper aims to fill this gap. It follows the tradition of Baker, Gibbs and Holmstrom (1994a, b)
and Lazear (2000) in using detailed analysis of the personnel records of one organisation to
examine responses to the incentive structure of the organisation. We exploit a very rich and
disaggregate dataset on the change in performance in an organisation which introduced a (pilot)
scheme of financial incentives. In 2002 a team-based incentive scheme was introduced into the
indirect tax assessment and collection agency of the UK government, Her Majesty’s Customs and
Excise (HMCE) 1 . We secured access to data from the personnel records and the management
information system for the incentive period and the equivalent prior period. These data are very
disaggregated, giving us the amount of time spent by each officer in each team each week on each
task and the yield collected. These rich data allow us to undertake a detailed examination of the
responses to the scheme.
The experiment in HMCE was a randomised control trial. We examine two treatment teams with
different incentive structures and one control team. Within the treatment teams, some of the tasks
were incentivised and some not. Exploiting the structure of the scheme and ther rich data we first
establish whether, at team level, the incentive scheme had an impact on performance against the
1
Now merged with the Inland Revenue to form HM Customs and Revenue.
3
targets set in the scheme, which were yield collection and time spent auditing. We then consider
two main potential sources of this: increased effort from individuals and a reallocation of
manpower by team managers to focus on the incentivised tasks.
We find that at the team level the incentive scheme did increase performance. At the individual
level, the nature of the experiment allows us to conduct a difference-in-difference-in-difference
analysis, exploiting variation across time, across teams and across tasks, at a worker-week-task
level. We find that the incentive structure raised individual tax yield and productivity. We then
identify efficient workers using data from before the implementation of the scheme. We investigate
the extent to which managers reallocate these workers to the incentivised tasks once the scheme
begins. We show that this happened, and happened disproportionately in one of the two treatment
teams. This team allocated more of all its workers’ time to the incentivised tasks, but
disproportionately reallocated the time of its efficient workers. We show that this reallocation was
the more important contributor to the overall outcome. The increase in individual yield collected
was essentially the same in the two incentivised teams. But one treatment team engaged in
reallocation to a significantly greater degree than the other, hit its targets and collected the bonus.
The paper finishes with some speculation about why one team engaged in this strategic behaviour
to a greater extent than the other. We find that on average, office managers were younger in this
team, and had less experience in the organisation. The overall team (division) manager was also
much younger compared to the other teams and had less experience. It may be that the different
strategic response of managers arose because the managers in the more successful team were
motivated by career concerns.
The next section of the paper describes the organisation we study, and provides some detail on the
incentive scheme. Section 3 describes the data and section 4 presents the results. Section 5 offers
some speculations as to why one team was more proactive in strategic task management.
2. The incentive scheme
HMCE is responsible for the collection of a wide range of excise duties and other indirect taxes
such as VAT in the UK. The impetus for the introduction of a pilot incentive scheme in the
organization was political, originating in the White Paper “Modernising Government” (1999), and
followed up in the Makinson report (2000) for the Public Sector Productivity Panel. This advocated
4
team-based incentives for frontline government workers. The ideas in the report were taken on by
three government agencies including HMCE. The scheme was implemented in twelve trial sites and
ran for nine months from April 2002 through December 2002.
a) Teams
The scheme was a team based incentive scheme. The scheme defined teams at the level of a
division, so each team comprised a small number of offices. Targets were devolved by the
divisional manager to office managers. Bonuses were set for achievement of various targets.
Performance with respect to these targets, and thus payment of the bonus, were evaluated at the
level of the whole division. If a team met its target, all workers in the team would receive the bonus
even if one or more locations did not reach their target. Complementarities in production seem
likely at office level, and so an office would be a team à la Holmström (1982). However, targets
were set and performance assessed at the level of a division, so creating interdependencies among
the offices in the same division.
The trial teams were paired and combined with a blind control team. The staff in the control team
did not know that they were monitored as the control group. We analyse the impact of the scheme
in two revenue gathering and tax assessment teams (known as “VAT Assurance” divisions) plus
their matched control, because the activities and outcomes for the set of activities undertaken in
these teams are more straightforward to quantify and analyse than those of some of the other teams
which were also subject to the trial. In these teams two different reward schemes were
implemented: in one team the bonus paid was equal across all team members (this team is referred
to as Team 2), and in the other the bonus varied according to the individual’s grade (Team 1). Team
1 comprised 154 workers, located in three offices; team 2 had 158 workers, located in six offices,
and the control team was somewhat larger with 281 workers.
b) The Bonus structure
A team became eligible for the bonus after meeting all its other business objectives and targets. The
bonus was then paid on achievement of the incentive targets. Each incentive target consisted of a
baseline target plus a ‘stretch’, set at 5% of the baseline target. The maximum bonus was paid if the
team achieved all the incentive targets in full. In cases where a team failed to meet all incentive
targets in full but met the baseline target and at least 50% of the stretch on all of the incentive
5
targets, the bonus was 50% of the total bonus. In cases where the achievement on any incentive
target fell below 50% of the stretch no bonus was paid.
Not all tasks were incentivised for the treatment teams. The work done in a VAT assurance division
can be split up into five activities (see table A1). The main activity is trader audit. Audit includes
the examination of trader’s records and recovering any unpaid tax. The other activities are rather
general (they include other trader work, non-trader work, non-core work, and work on
unregisterable entities). Trader audit work itself splits into non-VAT work (excise duties), work not
assigned to a particular trader, and audit work. The audit work is logged under specific types of
businesses, known as trader groups. HMCE operates a sophisticated risk assessment system to
better focus resources where it is thought most likely that insufficient tax revenue is declared and
collected. Businesses are allocated to these trader groups. Examples of groups are “low risk”,
“branches”, “large traders” and “exceptional risk”.
Each of the incentivised teams had five incentive targets. Two of the targets were associated with
achieving a specific number of audits and the other three targets were related to increasing the
amount of tax revenue. The incentivised targets for the number of audits related to two trader
groups: “new registrations” and “exceptional risk”. The incentivised trader groups for the tax yield
were these plus another group – the “large traders”.
The impact of a worker’s effort on the two incentivised tasks – number of audits and tax yield - is
rather different. For the number of audits, clearly effort directly maps onto the outcome. But for
yield recovered, the outcome will also depend on the nature of each case, and is less certain a
priori. Both incentivised outcomes are measured with a high degree of precision.
c) Issues identified in the literature
The team-based incentive scheme was intended to both raise workers’ effort and to foster
cooperation across offices. A number of important issues arising from the introduction of financial
incentives to affect workers’ productivity have been identified. The measurability of performance
and the multi-dimensionality of tasks have an important role in determining the effectiveness of
incentives to raise performance. Holmstrom and Milgrom (1991) show that in multi-tasking
contexts, when outputs are measured with different precision, the use of high-powered incentives
can be suboptimal for the principal. In particular, if one outcome reflects multiple dimensions of the
6
agent's effort, the prospect of the agent diverting his effort away from the less accurately measured
tasks makes the principal weaken the overall incentives. Performance measures can be imprecise
not only because they are affected by factors which are unrelated to workers effort – in this case
both principal and agents have the same information on the outcome - but also because there is
asymmetry of information. This causes a moral hazard problem in that the agent is able to
manipulate the performance measures in order to maximise his private rewards.
Courty and Marschke (2004) evaluate the provision of incentives in a federal Job Training Program
and show that the incentive scheme triggered undesired responses. They demonstrate that training
agencies timed graduations date to increase their performance outcomes and to maximize their
inter-temporal award and this reduced program efficiency, in that enrollees were prematurely
graduated and ended up with lower earnings. Healy (1985) and Oyer (1995,1998) analyse the effect
of bonus awards for managers and salespeople. They show that the discretion that mangers and
salespersons have in determining earnings and sales gives them the opportunity to game the system
and distort performance measurement to their personal advantage. Mangers could postpone or
anticipate accrual earnings, and salespeople could affect the data of purchase of an item in order to
reach a quota of performance and get a bonus. Steenburgh (2005) evaluates the effects of lump sum
bonuses on salesperson behaviour and finds that lump sum bonuses have some counterproductive
effects. In sum, the discretion that workers have over how they allocate time to different activities
or on how they can influence the performance measure may have some unexpected and undesired
consequences for the scheme designers. We investigate this issue here.
3. Data
The scheme ran from April 2002 until December 2002. We use data from the agency’s performance
management system and personnel records. We restrict our worker-level analysis to continuously
employed frontline staff. These are present in the first month of observation and the last month of
the scheme, register time on audit, and collect tax. There are 129 such workers in Team 1, 124 in
Team 2, and 197 in the control team. The remainder of the continuously employed staff were
clerical workers and managers.
There is little variance across teams in gender mix or average age. Typically, workers have spent
about 4 years in their current job band. Little overtime is worked, and most of the workers are full7
time. The average salary is approximately £20,000 per annum. The average potential bonus for
members of Team 1 was £688 and for Team 2 was £681, which is approximately 3% of mean
annual salary.
We have data at a very disaggregate level: we have records for each worker, in each week, and in
each trader group. We examine a set of outcomes: the time allocated by each worker to each trader
group, total yield per worker per trader group, positive and negative yield per worker per trader
group, and productivity (defined as yield relative to time allocated). We do not have a direct
measure for the number of visits to businesses (for audit, this is the incentivised outcome), so we
use the time spent on each trader group as a proxy. Yield is split into positive and negative as this
corresponds to the Customs and Excise categories of under- and over-declared tax.
We first briefly describe the raw data on outcomes 2 . All three teams spend more than half their
time on trader audit, a significant amount of their time on non-trader work and smaller amounts of
time on the other work activities. Of the 5 work activities, time on trader audit contributes to the
event target, whilst time on the other activities does not. The mean time spent on trader audit
activity was 8.2 hours per week per trader group per worker. On average, workers allocated time to
around 3 different trader groups per week; the highest number worked on was 9 per week. There
was a great deal of variability in time allocated across teams, workers, weeks, and trader groups.
The incentivised groups - new registrations and exceptional risk traders - account in the
incentivised period for 30% and 24% respectively of total audit time allocated in Team 1, 36% and
29% in Team 2 and 34% and 26% in the control team.
After completing an investigation, workers assign a result code to describe the outcome (for
example, over-declaration or under-declaration) and attach a monetary value to the outcome. Yield
can be positive or negative. We analyse positive outcomes and negative outcomes separately, as
both provide information on behaviour. The mean positive yield collected is £301,811 per worker
per week per trader group. However, this figure reflects the presence of some very large outlier
values; the median positive yield is only £4,591. These outliers mean that we carry out most of the
yield analysis on medians or using trimmed samples. The largest share of positive yield comes from
the exceptional risk trader group in all three teams, accounting for 38% in Team 1, 65% in Team 2
and 40% in the control team. New registrations and high risk groups follow in order of magnitude.
2
A detailed discussion of the outputs is available at <link to report>.
8
The distribution of negative yield across VAT trader groups is similar to the distribution of positive
yield. Table A2 provides a summary.
4. Estimation results
We first briefly consider the impact of the scheme at team level and then evaluate the impact on
individual performance and task allocation. The team level analysis addresses the overall impact on
performance of the incentivisation in the three teams. Further, the rich and disaggregate micro data
allows us to probe deeper into how this overall impact came about – through greater effort from
individual workers, and/or through smarter management.
a) Team level analysis
The research design involves variation in incentivisation across time, treatment status, and trader
group. So we perform difference-in-difference analysis across time and team, and difference-indifference-in-difference across time, team and trader group. In tables 1a-c, we calculate the
difference in the median of each of the five outcomes (time, total yield, positive yield, negative
yield and productivity over time. We use the median because of the very large outliers. We
consider outcomes for continuous frontline workers and for trader audit work. Results are shown
for all trader groups (table 1a), incentivised trader groups only (1b) and non-incentivised trader
groups only (1c).
Table 1a shows the median time spent on audit work across all trader groups increased for all three
teams, but increased more for Team 2. The median total yield and positive yield increases for the
two treatment teams, but not for the control team, and so did the median productivity. The median
negative yield did not change over the period. Table 1b shows that for incentivised trader groups
the results were somewhat different: only in Team 2 did the median time spent on auditing the
incentivised trader groups increase. The median total yield and the positive yield increased in both
treatment teams but substantially more in Team 2 (3 and 5 times more than in Team 1 respectively).
Productivity also increased more in Team 2 than in Team 1. In Table 1c, which shows the results
for the non-incentivised trader groups, Team 2 decreased time spent auditing these trader groups,
whereas Team 1 and the control team spent more time on them. The median total and positive yield
and also productivity increased only in Team 1 for these activities.
9
Tables 1d-f show the difference-in-difference estimates 3 . Table 1d reports on all trader groups
combined. The results suggest that both treatment teams increased their performance on all
outcomes relative to the control team, apart from negative yield. Focussing on incentivised trader
groups (table 1e), Team 2 increased time auditing the incentivised trader groups more than the
control group whereas Team 1 did not. Total yield, positive yield and productivity increased in both
treatment teams relative to the control team, but the increase was more substantial for Team 2. Note
that productivity on all trader groups (table 1d) increased more in Team 1 than in Team 2, but that
productivity on incentivised trader groups (table 1e) increased more in Team 2 than in Team 1. For
the non-incentivised trader groups, the scheme had no effect for Team 2 for yield and productivity,
and time decreased in Team 2 relative to the control team. The opposite is true for Team 1, which
increased the time spent on non-incentivised trader groups relative to the control team and also
increased the revenues from investigations and productivity on these trader groups.
Table 1g presents the difference-in-difference-in-difference results for the incentivised trader
groups (labelled M) relative to non incentivised trader groups (nonM) in the two treatment teams.
The results indicate that in Team 1, the incentive scheme had a negative impact on the time spent
on incentivised trader groups: Team 1 allocated more time to non-incentivised trader groups. The
opposite happened in Team 2 where the median time allocated to incentivised trader groups
increased. Median total yield and positive yield increased for the incentivised trader groups relative
to the non-incentivised trader groups in both teams, though the increase was more substantial for
Team 2. Productivity on incentivised trader groups relative to non-incentivised trader groups
decreased in Team 1 and increased in Team 2. Hence the incentive scheme had a positive impact on
productivity from incentivised trader groups only in Team 2 4 .
3
We do not calculate standard errors as we only have three team-level observations.
4
We also investigated whether the staff with different characteristics respond in different ways to the scheme, using
matching analysis. We matched staff in treatment teams with staff in the control team by observable characteristics
(grade, age, gender). This allows us to separate workers into groups, which are homogeneous in terms of part-time/fulltime status, age, gender and pay and investigate whether workers with certain characteristics get a higher outcome
compared to those workers without those specific characteristics. However the findings suggest that the specific worker
characteristics do not have any consistent and clear effect on the median worker level difference in outcomes between
the treatment and the control groups.
10
b) Individual level analysis – greater effort?
The data allow us to model the output of each worker in each trader group. We assume that output
is affected by personal characteristics, the type of trader groups visited, the type of team the worker
is in (which reflects the local market) and the incentive scheme. We first difference the outcome
variables between the nine months of the scheme and the corresponding nine months of the year
before, for each worker and each trader group. This removes all time invariant characteristics such
as the worker’s ability, the work environment and any invariant features of the local business
market. We regress this worker-trader-group level difference on a set of individual characteristics X
(years in grade, age, gender, part-time/full-time status, job band), dummies for which team the
individual is in, an incentivisation dummy (which varies by trader group), and dummies for trader
groups. The key parameter of interest is the incentivisation dummy. For outcome y, worker i and
trader group g we estimate:
Δyig = β.X i + δ.Team i + α .Incentive g + γ.G g + εig
Table 2 presents the results for four different specifications: with and without non-VAT outcomes,
and with and without trimming the outliers 5 . For each specification we present three sets of
estimates: with both treatment teams combined, controlling for team membership with intercept
dummies and then two from estimation separately for the treatment teams. We present the results
for the incentivisation variable only, with robust standard errors.
For specification 1, the results combining both teams show incentivisation had a positive effect on
all five outcomes, though none of the coefficients are statistically significant. If we consider the two
teams separately, the coefficient on time is negative for Team 1, although not statistically
significant, and positive and significant for Team 2. For total, positive and negative yield the
coefficient estimates are positive for both teams, but again not statistically significant from zero for
any of the three regressions.
5
Additionally, we carried out median regressions, not presented here. The regressions trimmed of outliers, shown in
Table 4, produce somewhat similar coefficients to the median regressions. In particular, we analysed regressions with
variables indicating the interaction between the incentivisation dummy and the worker-specific variables (years in
grade, age, gender, part-time/full-time worker and job band). These did not show any interaction effects, indicating no
differences in responses across different grades and gender of worker.
11
The second specification omits the non-VAT and non-trader audit categories. The non-VAT
category includes a range of trader groups, for example, tobacco duties and international trade
exports, that take up very little of the teams’ time but count for a very substantial amount of
positive yields collected. Hence, it is likely that this reflects a very different production function –
tax revenue received as a matter of course with little investigation required. The results from the
regressions for both teams again show a positive incentive effect for four out of the five outcomes,
but these differences are not statistically significant. If we consider the two teams separately, the
coefficients on time are very comparable to those in the previous specification. The incentivisation
coefficient is negative for Team 1 but not statistically significant and is significantly positive for
Team 2.
In the third specification, we exclude outliers: trimming 5% at the top and bottom of the yield
distribution. There is no upper boundary on the distribution of yields, and the highest recorded
yield in the worker-week-trader group data is £284m. But the mean is only £54k and the median
value is zero. We therefore need to explore the effect of incentivisation leaving out the outliers 6 .
This makes a substantial difference to the precision of estimation. The results for both teams show
a significant positive effect for yield, positive yield and negative yield. The first two have large
coefficients: both total yield and positive yield per worker in both teams increased on average by
more than £24k for the incentivised trader groups in the incentivised period relative to the previous
period. When we consider the two teams separately, the scheme has a positive and significant effect
on total yield and positive yield for both teams. Productivity increased more in Team 1 than in
Team 2, and is statistically significant only for Team 1.
In the fourth specification we both exclude outliers and omit the non-VAT and non-trader audit
trader groups. This is our preferred specification. The results are very similar to the third
specification, both in terms of coefficients and standard errors. Again, the incentive scheme has a
statistically significant positive effect on total yield and productivity for both teams.
To summarise, our results on individual workers indicate the effects of the scheme are as follows:
•
The treated teams spend more time on all trader groups and recover more yield than the
control team.
6
For yield, positive yield and negative yield, this affected 292 out of 2935 observations. For productivity, this affected
294 out of 2935 observations.
12
•
If we distinguish between incentivised and non incentivised trader groups, workers in Team
2, on aggregate, increased the time spent on incentivised trader groups relative to non
incentivised trader groups. This is not true for workers in Team 1, who decreased the time
spent auditing the incentivised trader groups.
•
Total yield and positive yield increase for the incentivised trader groups relative to the nonincentivised trader groups in both teams, the increase being larger for workers in Team 2.
•
Productivity on incentivised trader groups relative to non-incentivised trader groups
increases in Team 2 but decreases in Team 1.
•
There is no clear pattern relating individual characteristics to performance improvement.
This is perhaps not surprising given we are looking at first differences as the dependent
variable.
•
For the time outcome, the analysis at individual level gives similar results to the aggregate
analysis at team level. Individuals in Team 2, on average, increase their time spent on
incentivised trader groups by 38 hours in the incentivised period compared to the previous
period, holding all other variables fixed. This does not happen in Team 1, where
individuals, on average, spend 5 hours less on incentivised trader groups, though the result
is not statistically different from no change.
•
For the total yield and productivity outcomes results at individual level differ from results at
team level. Individuals in both teams increase total yield and productivity for the
incentivised trader groups. Individuals in Team 1 increase total yield on average by £25k
and individuals in Team 2 by £29k. The increase in productivity in incentivised trader
groups is larger in Team 1 than in Team 2: yields on an incentivised trader group per hour
increase on average by £71 in the nine months of the incentive scheme in Team 1 and £27
in Team 2, though results are statistically significant only for Team 1.
These results show that, at individual level, productivity increases more in Team 1 than in Team 2.
However, at team level Team 2 performs better than Team 1 in all outcomes for the incentivised
trader groups. This difference in response at individual and team level suggests that the strategy at
team level differed between the two treated teams. An important part of team strategy is the
allocation of workers across trader groups. Managers can strategically allocate more efficient
workers to incentivised trader groups in order to meet the targets. In what follows, we investigate
how the allocation of efficient workers differs across the two treated teams and the control team.
13
c) Strategic task allocation – smarter management?
The richness of the data we have allow a detailed investigation of the allocation of individuals to
different trader groups, and how this changes over time. We explore whether high productivity
individuals in the first period were allocated to the incentivised trader groups in the incentivised
period. We identify efficient workers as those whose productivity on VAT-trader groups in the
period before the scheme was in the top 25% of the productivity distribution of all workers in that
period. We analyse whether there is a statistically significant difference in the reallocation of time
on the incentivised and the non-incentivised trader groups by efficient workers and other workers in
each team.
We first test whether the mean change in time spent by efficient workers on incentivised trader
groups is the same as the mean change in time spent by the other workers on incentivised trader
groups. If true, this would imply no strategic task allocation. We then do the same for the nonincentivised trader groups. For incentivised (non-incentivised) trader groups, we expect the
efficient workers to have a relatively higher (lower) mean change (i.e. a more (less) positive
number) than the other workers. Table 3 presents these results. For the incentivised trader groups,
looking at all teams simultaneously, the results show that both types of workers increase the time
spent on incentivised trader groups over the two periods. However, the mean increase for efficient
workers is more than that of the other workers, and the result is statistically significant at 1%.
When we consider the three teams separately, all teams have a positive mean change though the
results are statistically significant only for Team 2 at 1% and marginally for Team 1.
In Team 2 the efficient workers substantially increase the time spent on incentivised trader groups
compared to other workers. The mean time spent on incentivised trader groups increases by 276
hours in the nine months of the scheme for efficient workers, whereas the other workers increase
the time spent on the same trader groups by 99 hours. In Team 1 efficient workers increase the time
spend on incentivised trader groups by 87 hours, whereas the other workers increase their time
spent on the same trader groups by 35 hours. The mean change is positive for efficient workers and
for the other workers in the control team, but is not significant.
For the non-incentivised trader groups for all three teams, we see that efficient workers spend less
time on these in the incentivised period. The other workers, however, increase the time spent on
non-incentivised trader groups.
14
When we analyse the three teams separately, we see that the efficient workers in Team 2 and the
Control team spend significantly less time on non-incentivised trader groups than the other workers
in their team. In Team 2 the other workers also decrease their time on non-incentivised trader
groups, but by less than efficient workers (an average decrease of 122 hours for efficient workers
versus 8 hours for the other workers). In Team 1 efficient workers increase their time spent on nonincentivised trader groups, though less than the other workers (an average increase of 29 hours for
efficient workers versus 54 hours for the other workers), though results are not statistically
significant.
We use a figure to represent teams’ reallocation of time between incentivised and non-incentivised
trader groups. We plot the change in hours on incentivised trader groups on the horizontal axis, and
the change in time allocated to non incentivised groups on the vertical axis. Moving up or down the
positively sloped 45o line indicates changes in total audit time, divided equally between
incentivised and un-incentivised groups. Moving up or down the negatively sloped 45o line
indicates reallocation of time within a given total. Any point in the space can be thought of as a
combination of a move along a positive 45o line and a negative 45o line. To be specific, the
horizontal dimension of a point (x, y) derives from an overall increase in time of (x – y)/2, and a net
reallocation of (x + y)/2. The latter is half of the vertical distance between (x, y) and the positively
sloped 45o line. The gross reallocation is twice that, namely the full vertical distance between (x, y)
and the positively sloped 45o line.
We have plotted the actual data for the three teams for efficient workers in Figure 1. These come
from the first row of the first block of Table 3 (horizontal axis) and the first row of the second
block (vertical axis). It is clear that efficient workers in Team 2 reallocate their time to incentivised
trader groups much more than in the other two teams. The figure makes the point in a very striking
way. We did not expect a net reallocation of time by efficient workers towards incentivised trader
groups in the control team. If we look at the net reallocation of other (less efficient) workers, shown
in table 3c, in Team 2 they increase the time spent on incentivised trader groups by 107, in Team 1
they decrease the time spent on incentivised trader groups by 19 hours and in the control team they
increase by 19 hours. Hence workers in Team 2 work more strategically: they swap their time from
non-incentivised trader groups to incentivised trader groups and the swap is much greater for
efficient workers. This is confirmed formally in the lower half of Table 3 (tables 3d and 3e test the
differences shown in table 3c).
15
Overtime hours increased very little during the scheme in both teams, so that the increase in total
audit time comes from other non-audit activities. Relocation to tasks may have had a substantial
impact on team level performance. Strategic task allocation is less clear in Team 1 where efficient
workers increased their time more than other workers on all trader groups, but divided it between
both incentivised and non-incentivised trader groups. As noted above, the incentive scheme took
account of multi-task issues and teams had to fulfil base targets for all their activities to be eligible
for the bonus. This meant that teams could not simply direct all effort at the incentivised tasks, but
had to make sure that the other jobs were done. This made the optimal reallocation of worker time
more subtle. It appears that Team 2 made a better job of this. Ex post we know that Team 2 hit all
its targets, but Team 1 missed the target on events for the new registrations trader group and also
the yield on this trader group, while it hit all other targets.
We use worker-trader group regressions to confirm this reallocation at individual level. We regress
the worker level change in time between the incentivised period and the previous period on the
incentivisation dummy and 11 interaction dummies, controlling for the same variables as in
specification 2 of Table 4. The 11 interactions are the trader group dummies interacted with an
efficient worker marker. The coefficient on these variables gives the worker-group level change in
time for efficient workers on the trader groups.
The results are presented in Table 4; the incentivised trader groups for the time outcome (new
registrations and exceptional risk) are highlighted in the table. The results for efficient workers in
Team 1 are mixed: they increase time spent on exceptional risk traders by 63 hours, but decreased
time spent on new registrations by 15 hours over the period, though the result is not statistically
significant. They also increased time spent on a set of non-incentivised trader groups, though the
results are not generally significant. Efficient workers in Team 2 increase their time spent on
exceptional risk traders by 123 hours in the incentivised period and on new registrations by 44
hours. They decrease time spent on all other trader groups, apart from deregistered and insolvent,
though coefficients are statistically significant only for the deregistered trader group. These results
confirm that Team 2 has a more strategic approach in allocating efficient workers to the
incentivised trader groups.
16
d) Overall decomposition of the incentivisation effect
We now examine how much of the change in a team’s yield is due to strategic reallocation of
workers and how much to greater effort. We denote the productivity of worker i in trader group j in
period t as ψijt. It is measured as the ratio of total yield (Y) collected by that worker in that trader
group to the time spent (T) on that trader group by that worker. The worker’s overall productivity in
period 1 is the sum of this over all trader groups, weighted by the share of that worker’s time on
that trader group, τijt: ψ it ≡ ∑ (Yijt Tijt )*τ ijt .
j
In the pre-incentive period, t = 1, we assume that this represents each worker’s ability and base
effort. We assume that the change in performance of an individual worker is due to extra effort, so
that ψi2 = ψi1 + πi. Productivity at team level in period 1 and 2 is the sum of weighted
productivities over all trader groups and all workers. With a little manipulation, we can see that the
increase in total yield between the two periods can be decomposed as follows:
ΔY = Y2 − Y1 = ∑
i
∑ (T
ij 2
j
− Tij1 )ψ ij1 + ∑
i
∑ (ψ
ij 2
−ψ ij1 ) Tij 2
j
The first term represents the change in time allocated by worker i to group j. This could be further
split down into a change in the total audit time of i, Ti1 to Ti2 and a change in the way that that is
allocated across trader groups, τij1 to τij2. The second term captures the change in productivity for a
given time allocation. We calculate this decomposition for our data and report the results in Table
5. Note that some workers spent time on incentivised trader groups only in one of the two periods
and also collected yields on the same groups only in one period. So we lose these unbalanced
observations. In the table we report the change in total yield we obtain using balanced observations
(the across team patterns in the change in total yield including all the observations are very
similar) 7 .
The results show that the fraction of the total change in total yield accounted for by the reallocation
of time is 75.3% in team 1, 93.5% in team 2 and 85.2% in the control team. These results confirm
our findings for the strategic reallocation of workers: in all teams the change in total yield is
7
We lose 6.7% observations in team 1 in period 1, 30.4% in the same team in period 2; 5.3% and 27.7% in team 2 in
the first and second period respectively; and 11.2% and 31.6% in the control team.
17
explained mostly by an increase in the time spent on incentivised trader groups. This effect is
particularly strong for team 2.
5. Conclusion
We have identified that the pilot incentive scheme raised individual productivity, and led to the
reallocation of efficient workers towards the incentivised tasks. This reallocation was the more
important contributor to the overall outcome. The increase in individual yield collected was
essentially the same in the two teams. But team 2 engaged in reallocation to a significantly greater
degree than team 1, and it hit its targets and collected the bonus. In this final section we consider a
number of possible explanations for the difference in strategic behaviour that we have identified.
Staff composition might be such that, for example, workers in team 2 were on average more
experienced than team 1 and hence better suited to reallocate their effort towards the incentivised
trader groups. Examining the profile of age, time spent in the organisation and time in grade, we
find these are similar in the two teams. Examining the grade (job band) profile across the teams, we
find that both teams have approximately the same percentage of workers in job bands 6 and 7 (7580%). Team 2 has more workers in lower, clerical, job bands and team 1 has more senior workers
(in job bands 8 and 9). Hence it is not the case that workers in team 2 are more experienced or work
in higher grades.
The result could be due to a difference in the managerial strategy across the two incentivised teams,
which existed prior to the introduction of the scheme. One team may have a more flexible task
assignment strategy because workers are used to dealing with different trader groups, so that they
have enough experience to be effectively reallocated across trader groups. This could in turn be due
to the initial choice about how the work was organised in offices or it could be due to the fact that
some trader groups are concentrated in certain areas, so that workers working in those areas need to
focus on a limited number of trader groups and hence there is less scope for reallocation. In both
cases a more/less flexible task allocation strategy is not related to the incentive scheme. However,
we find little difference in the number of trader groups that each team works with.
The strategic task reallocation could also be due to differences in managers in the two incentivised
teams. We know that the bonus scheme in team 2 is a flat bonus so, relative to managers in team 1,
18
managers in team 2 receive a lower bonus. If managers were only motivated by financial incentives
we would expect managers in team 1 to be more proactive in strategic behaviour, but we actually
find the opposite. But more intrinsic incentives may be at work, and so we examine how managers’
observable characteristics differ in the three teams. We can distinguish office managers (job band
9) from division managers (job band 11, only one per division). On average, office managers were
younger in team 2, they had been in that job band for fewer years and they had less experience in
the organisation. The division manager in team 2 was much younger than in either the control or
team 1, and had less experience. One possible explanation for the different strategic response of
managers in the two incentivised teams is that managers in team 2 may have been more driven to
perform well - and so to reallocate efficient workers - for longer term career concerns.
19
Table 1: Team Level Analysis
Table 1a
Table 1b
Table 1c
For all trader groups
For all incentivised
For all non-incentivised
Differences between incentivised & pre-incentivised
period
Differences between incentivised & pre-incentivised
period
Differences between incentivised & pre-incentivised
period
Variable
Variable
Variable
Team 1
Team 2
C
Total Time
50.5
103.35
41.5
Total Yield
67193
73441.5
Positive Yield
84983
Negative Yield
Productivity
Team 1
Team 2
C
Team 1
Team 2
C
Total Time
-3.8
73.89
-1.5
Total Time
13.5
-13.4
1.099
0
Total Yield
10901
31725
0
Total Yield
6406
0
0
80877
0
Positive Yield
8370
46550.5
0
Positive Yield
6406
0
0
0
0
0
Negative Yield
0
0
0
Negative Yield
0
0
0
137.54
120.13
0
Productivity
58.70
89.78
9.17
58.24
0
0
Productivity
Table 1d
Table 1e
Table 1f
Difference-in-Difference
Difference-in-Difference
Difference-in-Difference
Variable
T1-C
T2-C
Variable
T1-C
T2-C
Variable
T1-C
T2-C
Total Time
9
61.85
Total Time
-2.3
75.40
Total Time
12.4
-14.5
Total Yield
67193
73441.5
Total Yield
10901
31725
Total Yield
6406
0
Positive Yield
84983
80877
Positive Yield
8370
46550.5
Positive Yield
6406
0
Negative Yield
0
0
Negative Yield
0
0
Negative Yield
0
0
137.54
120.13
49.53
80.61
58.24
0
Productivity
Productivity
Productivity
Difference-in-Difference-in-Difference
Variable
M*(T1-TC) – nonM*(T1-TC)
M*(T2-TC) – nonM*(T2-TC)
Total Time
-14.7
89.89
Total Yield
4495
31725
Positive Yield
1964
46550.5
Negative Yield
0
0
-8.71
80.61
Productivity
Notes:
M indicates incentivised
Data are monthly differences per officer and trader group summed over the 9-month period of the scheme
20
Table 2: Estimating the incentive effect at individual-trader group level
Specification 1 a
Both teams
Team 1
Team 2
Specification 2 b
Both teams
Team 1
Team 2
Specification 3 c
Both teams
Team 1
Team 2
Specification 4 d
Both teams
Team 1
Team 2
Time
Yield
Positive yield
Negative yield
Productivity
16.662
(12.538)
-4.922
(12.373)
37.894 ***
(14.520)
862,226.531
(652,598.418)
136,162.869
(98,686.498)
978,548.109
(860,264.244)
879,553.131
(653,435.097)
145,919.115
(98,767.889)
1,001,806.908
(860,344.116)
-17,326.599
(12,221.249)
-9,756.246
(7,560.350)
-23,258.800
(21,407.965)
382.772
(3,608.746)
239.004
(470.589)
271.321
(408.845)
14.010
(12.902)
-4.581
(12.420)
37.937 ***
(14.541)
435,750.800
(412,809.891)
47,088.951
(42,239.992)
896,445.946
(860,973.361)
453,579.263
(414,757.543)
57,081.507
(43,773.840)
919,626.879
(860,979.605)
-17,828.463
(12,356.234)
-9,992.555
(7,575.689)
-23,180.933
(21,053.030)
40.633
(144.382)
166.134
(249.878)
58.324
(151.494)
24,517.457***
(5,292.030)
25,815.132***
(7,004.909)
29,781.494***
(6,927.112)
24,629.887***
(5,838.805)
25,696.292***
(7,864.822)
28,954.846***
(7,314.759)
-633.594***
(226.192)
-562.030**
(218.070)
-899.411***
(297.831)
28.494
(23.125)
70.981**
(32.555)
26.951
(22.341)
22,360.027***
(5,254.457)
25,180.406***
(6,873.621)
28,775.089***
(6,879.612)
22,154.517***
(5,842.350)
25,583.105***
(7,827.547)
28,803.199***
(7,288.997)
-585.759**
(236.258)
-549.367**
(219.232)
-879.165***
(296.545)
32.439
(23.020)
70.908**
(32.484)
27.019
(22.287)
* significant at 10%; ** significant at 5%; *** significant at 1% Significant coefficients (5% or better) are in bold.
a
This specification controls for the following variables: team membership, years in grade, age, mean
age squared, gender, part-time/full-time worker, job band, all trader groups.
b
This specification controls for the same variables as above, but excludes the non-VAT and non-trader
audit trader groups.
c
This specification controls for the same explanatory variables as specification 1, but excludes the
outliers of the dependent variables (defined as the top and bottom 5%).
d
This specification excludes the outliers of the dependent variables and omits the non-VAT and nontrader audit trader groups among the explanatory variables.
21
Table 3: Strategic task reallocation
Individual level difference in time spent per month on VAT trader groups during
the 9 months of the scheme relative to the same period prior to the scheme:
(a) Incentivised Trader Groups
Efficient workers
Others
All teams
Team 1
Team 2
Control team
131.94
56.89
t = 2.98
87.12
35.02
t = 1.50
275.91
98.84
t = 3.46
48.51
45.92
t = 0.06
Significant at 1%
Significant at 10%
Significant at 1%
Not significant
(b) Non-incentivised Trader Groups
Efficient workers
Others
All teams
Team 1
Team 2
Control team
-63.56
26.04
t = -4.30
29.31
54.05
t = -0.741
-122.17
-8.31
t = -2.66
-89.29
28.73
t = -3.75
Significant at 1%
Not significant
Significant at 1%
Significant at 1%
Reallocation between incentivised and non-incentivised trader groups:
(c) Efficient workers vs. others
Efficient workers
Others
All teams
Team 1
Team 2
Control team
195.50
30.84
t = 4.56
57.80
-19.03
t = 1.52
398.08
107.15
t = 3.91
137.79
17.18
t = 2.1386
Significant at 1%
Significant at 10%
Significant at 1%
Significant at 5%
(d) Team X vs. Team Y, reallocation of efficient workers only
Team X
Team Y
Team 1 vs. Team 2
Team 1 vs. Team C
Team 2 vs. Team C
57.80
398.08
t = -4.25
57.80
137.79
t = -1.13
398.08
137.79
t = 2.92
Significant at 1%
Not significant
Significant at 1%
(e) Team X vs. Team Y, reallocation of other workers only
Team X
Team Y
Team 1 vs. Team 2
Team 1 vs. Team C
Team 2 vs. Team C
-19.03
107.15
t = -2.96
-19.03
17.18
t = -1.0241
107.15
17.18
t = 2.21
Not significant
Significant at 1%
Significant at 5%
The number of efficient workers for each team is: 28 (out of 129) for Team 1, 29 (out of 124) for Team
2 and 35 (out of 197) for the control team. (Total=92 out of 450).
22
Table 4 – Time change per trader group
Efficient worker * New registrations
Efficient worker * Low risk
Efficient worker * Median risk
Efficient worker * High risk
Efficient worker * Exceptional risk
Efficient worker * (999) Large traders
Efficient worker * Corporate groups
Efficient worker * Branches
Efficient worker * Insolvent
Efficient worker * Deregistered
Efficient worker * Missing traders
Team 1
Team 2
-15.001
(21.992)
-3.071
(8.022)
14.049
(9.806)
-15.169
(20.790)
62.801 **
(25.398)
-14.579
(23.095)
0
(0.000)
0.495
(4.063)
9.547 **
(3.865)
3.002
(3.116)
10.339
(6.056)
43.716 *
(25.351)
-26.502 ***
(9.769)
-18.213 **
(8.882)
-45.998
(29.948)
122.785 ***
(42.559)
-55.919
(41.144)
-9.942
(6.982)
0
(0.000)
4.715
(3.299)
9.241 **
(3.895)
-0.232
(2.515)
23
Table 5 Decomposition of the change in total yield
N workers
A (change in time spent)
B (change in productivity)
A+B
A as % of the change in yield
B as % of the change in yield
Team 1
Team 2
Control Team
102
109
117
10,500,514
3,445,083
13,945,597
19,785,364
1,378,579
21,163,943
4,330,667
752,514
5,083,181
75.3 %
24.7 %
93.5 %
6.5 %
85.2 %
14.8 %
We have taken out the highest single yield data point in team 2 (£280m)
24
Figure 1: Strategic task allocation: change in time of efficient workers
Change in time spent on NI_TG
T1
29
49
87
Change in time spent on I_TG
276
-89
C
T2
-122
25
References
Baker, Gibbs and Holmstrom (1994a) 'The internal economics of a firm: Evidence
from personnel data', Quarterly Journal of Economics, 109, pp.881-921
Baker, Gibbs and Holmstrom (1994b) 'The wage policy of a firm', Quarterly Journal
of Economics, 109, pp.921-57
Besley, T. and Ghatak, M. (2005) Competition and Incentives with Motivated Agents.
American Economic Review 95 (3) 616-636.
Burgess, S., and M. L. Ratto (2003) “The role of incentives in the public sector: issues
and evidence”, Oxford Review of Economic Policy vol. 19 no. 2.
Courty, P., and Marschke, J. (2004) “An empirical investigation of gaming responses
to explicit performance incentives”, Journal of Labor Economics. Vol. 22, no.
2, pp. 23-56.
Dixit, A. (2002) “Incentives and organisations in the public sector: an interpretative
review” Journal of Human Resources, 37(4), pp.696-727.
Francois, P. (2000) “’Public service motivation’ as an argument for government
provision”,Journal of Public Economics,78, pp.275-299.
Healey, P., 1985, “The effect of Bonus Schemes on Accounting Decisins”, Journal of
Accounting and Economics, 7(1-3), p. 159-81.
Heckman, J.J., Heinrich, C., Smith, J., 2002, “The performance of performance
standards” Journal of Human Resources, 37(4), pp. 778-811.
Holmström, B. (1982) “Moral hazard in teams”, Bell Journal of Economics, 13, pp.
324-340.
Holmstrom B., Milgrom P. , ``Multi-task principal-agent analyses: Linear contracts,
asset ownership and job design'', Journal of Law, Economics and
Organisation, 7 (Special Issue), 1991, pp.24-52.
Lazear Edward, (2000) "Performance Pay and Productivity" American Economic
Review, 90:5, pp. 1346-1361.
Oyer, P., 1995, “The effect of Sales Incentives on Business Seasonality”, Unpublished
manuscript, Princeton University.
Oyer, P., 1998, “Fiscal year ends and nonlinear incentive contracts: the effect on
business seasonality”, Quarterly Journal of Economics, pp. 149-185.
Makinson, J. (2000) “Incentives for change. Rewarding performance in national
government networks”, Public Service Productivity Panel. HMSO.
Prendergast, C. (1999) “The provision of incentives in firms”, Journal of Economic
Literature, vol 37, 7-63.
Steenburgh , “Effort or timing: the effect of lump-sum bonuses”, Harvard Business
school marketing research papers no. 05-03, January 2005
White Paper “Modernising Government” (1999).
documents.co.uk
www.archive.official-
26
Appendix
Table A1: The incentive scheme
Activities of VAT
Trader groups
assurance office
Non-trader work
Other trader work
None-core
Unregisterable entity Trader audit work
9
divided into the following trader groups:
Non-VAT
Non-trader audit
New registration
Low risk
Medium risk
High risk
Exceptional risk
Large traders
Corporate groups
Branches
Insolvent
Deregistered
Missing traders
No. of audits
(time)
9
9
-
Tax revenue
(yield)
9
9
9
-
Notes
1. Non-VAT trader groups include (e.g.) duties and international trade exports
2. Non-trader audit includes all activities other than trader which are not assigned to a particular trader
audit
Table A2: Output statistics per officer, week, and trader group
Variable
Mean Std Dev Median
N
Time
8.24
7.67
6.5
70130
Pos. Yield (£)
301,811 4,430,267 4,591
17741
Neg. Yield (£) -3,485.73 39,457.77 0.00
17741
27