Building State Capacity: Positive vs. Negative
Incentives for Tax Compliance
Thad Dunning,∗ Fernando Rosenblatt,† Rafael Piñeiro,‡ and Felipe Monestier§
Research design and draft pre-analysis plan prepared for the Comparative Politics Workshop, University of
Pennsylvania, April 10, 2014. We are grateful to Guadalupe Tuñón for comments and research assistance, and to
seminar participants at Stanford University and the 10th meetings of the Experiments in Governance and Politics
(EGAP) group.
∗
University of California, Berkeley.
Universidad Diego Portales, Santiago, Chile
‡
Universidad Católica, Montevideo, Uruguay
§
Universidad de la República, Montevideo, Uruguay
†
1
To Penn CP workshop readers: I am circulating this research design and draft pre-analysis plan for
feedback and critique, relating to research we plan to undertake in Summer 2014. Though my main hope
is that you will help us refine the study design, I also want to try this out as a form of “results-blind
review.” Some scholars have advocated results-blind review for journals, as a way to limit publication
bias and data mining, boost publication of informative null results, and allow for elicitation of scholarly
priors about effect sizes. I am curious to see how this format works out in a CP setting and what kind of
discussion it generates. – Thad
Abstract. How can persistent obstacles to direct taxation in developing countries best be overcome? We
study a unique randomized policy innovation in Montevideo, Uruguay, in which the municipal
government raffles tax holidays to eligible taxpayers. Using over-time tax payment records and our own
survey data, we will assess the impact of holidays on subsequent tax compliance, as well as on citizens’
beliefs about the equity and fairness of the tax system. We also use an experimental intervention to study
the effects of informing non-winners about the tax holiday lottery—which has not been effectively
advertised by the government—allowing us to estimate multiplier and spillover effects of the program at
full scale. Finally, we also compare the effects of information about the tax holiday to the impact of
information about potential penalties for tax evasion, allowing us to assess the relative influence of
positive and negative incentives for compliance. Our findings may inform tax policy in other settings, as
well as our understanding of central barriers to direct taxation in a developing-country setting.
2
Contents
1
Introduction and Background
5
2
Theory
8
3
Empirical Strategy
3.1
A natural experiment: estimating effects of the tax holiday lottery . . . . . . . . . . . . . 13
3.2
Informational interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.3
4
5
13
3.2.1
Text of messages on tax bills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2.2
Parsing informational effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.3
Positive vs. negative incentives for tax compliance . . . . . . . . . . . . . . . . . 21
Treatment assignment and design matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Data collection
23
4.1
Administrative tax payment data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2
Household sample and survey design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.3
Other outcomes: call data from flyer recipients . . . . . . . . . . . . . . . . . . . . . . . 25
4.4
Power calculations and sample sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Analysis plan
28
5.1
Hypotheses and tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5.2
Additional survey questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.3
Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6
Conclusion: Relevance, Contribution, and Value of Research
33
7
References
35
3
List of Tables
3.1
Treatment Conditions and Sample Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
List of Figures
1
Power Calculations for Different True Effect Sizes (Two-tailed tests, binary outcome) . . . 37
2
Power Calculations for Different True Effect Sizes (One-tailed tests, binary outcome) . . . 38
3
Power Calculations for Different True Effect Sizes (Two-tailed tests, graded outcome) . . . 39
4
Power Calculations for Different True Effect Sizes (One-tailed tests, graded outcome) . . . 40
4
1
Introduction and Background
Developing countries face persistent obstacles to engendering tax compliance, a key facet of state capacity.
One central challenge is the difficulty of monitoring compliance and enforcing sanctions for non-payment
of taxes.1 Weak administrative capacity is often cited as the reason for low compliance, and also the reason
that many developing countries turn to easier-to-collect indirect taxes (export duties, sales and value-added
taxes) rather than direct taxation of income or assets. Underlying this account is a behavioral theory about
incentives for tax compliance—in particular, the theory that citizens weigh the benefits of tax evasion
against the threat of sanctions discounted by the probability of detection.
Yet, the positive benefits of paying taxes must also explain some of the variance in tax compliance.
In many developed countries, for example, the probability of detection for tax evasion is quite low—
yet many people nonetheless pay their taxes.2 The benefits of paying taxes may be material (e.g. a direct
individual or group return to taxes, in the form of public spending) or expressive or normative, for instance,
due to social preferences, equity considerations, altruism, or burden sharing.3 However, while scholars
have increasingly studied the latter factors as motivations for tax payment as well as tax policy, they have
perhaps less frequently studied the effect of positive material incentives for tax compliance. The benefits
of taxation for public spending play a key role in theories of redistributive voting.4 Yet such theories do not
tell us how benefits may influence actual compliance, nor are predictions about the relationship between
benefits and compliance typically put to empirical test.
These observations beg the question: do the perceived benefits of paying taxes shape tax compliance? And how do the effects of positive incentives compare to the impact of negative incentives, in the
form of sanctions for tax evasion? Given severe selection problems, it is typically challenging to answer
such questions in non-randomized observational studies. For example, benefits may be provided to more
politically mobilized taxpayers, who may also tend to pay their taxes; in political systems in which the
tax bureaucracy is already more effective, more benefits may be provided, thus confounding any apparent
1
See e.g. Bates and Lien (1985), Besley and Persson (2009); also Levi (1988) and Tilly (1990).
Andreoni et al. (1998: 819).
3
See e.g. Scheve and Stasavage (2012) on taxation as burden sharing in the wake of mass mobilization for war.
4
E.g., Meltzer and Richards (1981); Acemoglu and Robinson (2001, 2006); Boix (2003).
2
5
effect of benefits on compliance.
In this project, we study an unusual randomized tax holiday policy in Montevideo, Uruguay, which
can shed light on several aspects of the compliance problem. The innovative policy developed by Montevideo’s City Hall (Intendencia Municipal de Montevideo—IMM) seeks to improve tax compliance by
providing benefits for good tax-paying behavior. Since 2004, across four kinds of taxes (property, vehicle,
sewage, and head), the municipality of Montevideo has randomly selected taxpayers and—conditional on
a history of good taxpaying—rewarded them with a year free of tax payments. The rebate policy was initiated by the center-left government of the Frente Amplio in the context of an amnesty for many delinquent
taxpayers following the economic crisis of 2002. The idea was to counteract perceived negative incentives of the tax amnesty. As officials at the IMM have told us, the economic crisis generated a dilemma:
how to lower the burden for those under dire circumstances while at the same time continuing to promote
compliance. The lotteries were their answer.5
After almost ten years, however, no evaluation of the program’s effectiveness exists. Moreover,
somewhat surprisingly, citizens’ knowledge of this rebate lottery does not appear particularly widespread.
This provides an opportunity for us to study barriers and possible solutions to problems of engendering
tax compliance, for instance, whether positive (promise of rewards) or negative (threat of sanctions, fines)
incentives have a larger effect on boosting compliance. Our findings should have policy implications for
governments facing tax compliance problems, and also provide insights into basic motivations for tax
compliance, as well the broader social and political effects of the municipality’s rebate program.
We propose (1) to study the impact of the existing program, using our access to over-time tax payment records, as well as household survey data; (2) to evaluate the best means of improving the program’s
effectiveness, through informational interventions that increase citizens’ knowledge of the rebate lottery;
and (3) to study citizens’ motivations for tax compliance, in particular, whether positive or negative incentives have a greater effect on payment of taxes. The municipal government has granted us access to
over-time taxpayer records and is cooperating closely on our informational interventions, which will be
printed on actual tax bills and mailed to households.
5
In October 2013, the municipality announced a renewed amnesty for certain bad taxpayers, underscoring the difficulties of
cracking down on tax evasion and the potential importance of positive incentives.
6
(1) Impact of the tax holiday lottery. Winning the tax holiday lottery may inform eligible
taxpayers of the existence of the lottery and also shape their attitudes towards taxation,
thereby affecting subsequent tax compliance. We will use our panel of administrative data to
assess whether winning the lottery causes greater tax compliance over time. The lottery sets
up a straightforward comparison, among eligible taxpayers—those with good tax-payment
records—between winners and a randomly selected control group of eligible non-winners.
This natural experiment allows us to estimate the effect of winning the tax holiday benefit.
Yet, the impact of the program is likely greater than this comparison will suggest, because
knowledge of the program may influence citizens to bring their tax accounts up to date (to
become “good taxpayers”) in order to gain eligibility for tax holidays. We will use
randomized interventions to assess whether informing citizens, including bad taxpayers,
about the existence of the lottery affects subsequent tax compliance. Our household survey
data will allow us to distinguish between income effects and informational (learning about
the lottery’s existence) and attitudinal (perceived equity and benefits of taxation) effects of
winning a tax holiday. We are also interested in assessing variation in effects across different
types of taxes; and we will use administrative data to estimate spillover effects on neighbors.
(2) Improving effectiveness through informational interventions. The municipal
government of Montevideo appears to have done a poor job of promoting knowledge of the
tax rebate lottery. Our informational interventions, conducted with the cooperation of the
municipal bureaucracy, will allow us to compare the costs and benefits of a campaign to
increase awareness of the program. We also vary the form in which information is presented
to potential taxpayers, which will provide evidence on how best to maximize program impact
when the program is scaled up or transported to other settings.
(3) Motivations for tax compliance. Our informational treatments include a condition in
which taxpayers are reminded of possible fines and sanctions for non-payment of taxes. Like
our treatment informing taxpayers of a possible benefit due to the existence of the lottery,
7
this informational treatment will appear printed on a tax bill that is mailed by the municipal
government. We present the information so as to equate, to the extent possible, perceptions
of the typical size of rewards and punishments for tax compliance and non-compliance,
respectively. This allows for a direct comparison of the effect of positive and negative
material incentives for tax payment. We will also use additional informational interventions
that prime the social return to paying taxes and the social rationale for punishing tax evaders;
and we ask post-treatment questions about equity and fairness of the tax system, in order to
assess whether the lottery may boost compliance by shaping material incentives or by
influencing respondent’s attitudes towards taxation and the municipal government.
In the rest of this document, we discuss the theory according to which positive incentives may affect
tax compliance; and we discuss our design, including the basic empirical and data collection strategy, as
well as our plan for analysis of the data. As we discuss in the Conclusion, the innovative policy adopted in
Montevideo may well suggest avenues for constructing greater state capacity in other developing-country
settings.
2
Theory
In a standard expected utility model of incentives for noncompliant behavior, the decision to pay taxes
is a function of the benefit of tax evasion, minus the cost of punishment discounted by the probability
of being caught.6 Thus, expected penalties for non-compliance are critical to boosting compliance. Evidence from lab experiments suggests that the probability and size of a penalty for non-payment does have
a positive influence on compliance.7 Castro and Scartascini (2013) also corroborated the sensitivity of
compliance to the threat of sanctions in an Argentine municipality, using a field experiment similar to one
of our informational interventions described below. In their study, messages sent with municipal tax bills
reminded taxpayers of penalties for noncompliance; these messages boosted payment rates by over four
6
7
E.g., Allingham and Sandmo 1972).
Friedland, Maital, and Rutenberg (1978); Becker, Buchner, and Sleeking (1987); and Beck, Davis, and Jung (1991).
8
percentage points, relative to a control group.8 Thus, the threat of sanctions for noncompliant behavior
can clearly influence the rate of tax payment. If so, difficulties in detecting and punishing tax noncompliance may then explain high rates of tax evasion in many developing countries, where administrative and
bureaucratic capacities are often weak. Administrative weaknesses may also explain the choice over types
of taxes in many developing countries (as well as historically in today’s developed countries): instead of
difficult-to-collect direct taxes on income or assets, states have relied on relatively easy-to-collect export
taxes, or sales or value-added taxes, for which monitoring and enforcement costs are substantially lower.
However, this standard account also ignores potential perceived benefits of taxation to taxpayers—
e.g., the return in terms of public spending, either on individually targeted benefits or on public goods,
as well as social or normative reasons for paying taxes. Ignoring material and expressive benefits of
paying taxes may account for the mismatch between the standard expected utility model and evidence on
the extent of noncompliance. Indeed, though the expected costs of punishment for evasion often appear
quite small, people often still pay taxes, certainly in many developed countries but even in some lowand middle-income countries as well. As Andreoni et al. (1998: 819) have put it, “Many households
comply more fully than is predicted by [the expected utility] approach. Economists are just beginning to
grapple with the findings from other social sciences that could explain observed compliance levels, such
as a household’s sense of moral or social obligation to pay its taxes.” This point is sharpened by laboratory
evidence from, inter alia, Alm, Jackson, and McKee (1992), who find that when audit probabilities and
penalties are set at levels consistent with real-world tax systems, their deterrent effect is quite small.
If the threat of penalties does not (solely) determine tax compliance, what does? And what are the
implications of positive incentives to pay taxes for the construction of tax-collecting capacity in developing
countries? As discussed in the introduction, in this project we study the impact of one set of benefits for
tax compliance, which we compare to the impact of sanctions and threats of punishment. In particular, we
study the effect of the innovative randomized tax holiday program in Montevideo, which provides a year
free of asset tax payments to eligible taxpayers—i.e., “good” taxpayers who have been current on their tax
payments over the previous fiscal year. We also add an informational intervention to assess the impact of
8
Castro and Scartascini (2013).
9
priming perceptions of taxation’s social benefit.
Yet, how should we expect the possibility of such benefits to influence tax payment behavior? To
begin, it is useful to note that positive incentives to pay taxes may well not generate greater compliance. In
an expected utility framework that only includes the direct positive benefit of the tax holiday, the expected
costs of tax evasion may still need to be prohibitively high to generate compliance. For instance, under
the municipal rebate policy we study, good taxpayers are eligible for participation in a lottery that rewards
winners with one year free of tax payments, and they win with probability 1/10,000. Let y be the value
of an asset, t be the annual tax rate on that asset, and z be the unpaid amount on taxes due; in the case
of full nonpayment of a year’s taxes, z = ty. Furthermore, let c be the penalty for nonpayment and p the
probability of being caught in an audit. The expected utility of full nonpayment is thus
z − pc,
(1)
while the expected utility of paying the full year’s taxes this year (z) in order to gain eligibility for lottery
and possibly win a year free of taxes in the next year is, without discounting,
1
z − z.
10, 000
(2)
The latter is only larger than the former when
pc >
19, 999
z.
10, 000
(3)
If p < 1, then c must be larger than about 2z for tax payment to be optimal. In words, the cost of the
penalty discounted by the probability of punishment must be essentially as large as two full years of taxes
due on the asset.9
A priori, this might make it seem unlikely that a positive benefit to paying taxes, in the form
9
Here, there is no discount factor, so in equation (2) the benefit of winning the lottery and receiving a tax holiday in the next
year is just (1/10, 000)z, rather than, say, β(1/10, 000)z, with β ∈ (0, 1). With discounting, the left-hand side of (2) is slightly
larger, so the discounted cost of punishment must be even bigger to elicit compliance.
10
of a randomized tax holiday, would have any effect on tax compliance. Yet, this conclusion does not
necessarily seem warranted. In the first place, and perhaps most trivially, the value of z varies for different
taxpayers. For instance, some taxpayers may be only a bit delinquent on their tax payments, such that
the cost of compliance may be less than a year’s tax on assets (z < ty). Also, z also obviously varies
as a function of asset values. This might lead us to expect heterogeneous effects of an intervention to
inform taxpayers about a randomized tax holiday. In particular, effects could be more pronounced on the
margin for taxpayers with less valuable assets, and those who are less delinquent on their taxes. Another
possibility is lies in the difference between subjective and objective probabilities: people may overestimate
the probability of detection p, making it more likely that condition (3) will bind.10
However, a second set of considerations goes outside a framework based strictly on the material
benefits and (expected) cost of tax evasion. For example, non-payment of taxes might generate a “conscience” cost, which can shift the conditions under which tax evasion is optimal.11 Even beyond this
observation, tax policy might be expected to generate several kinds of benefits associated with payment
of taxes. In the first place, as noted above, paying taxes may generate a direct individual return, such
as the expected benefit due to the tax holiday lottery we study here. It may also provide an expressive
benefit associated with the social return of taxation. And finally, the way that benefits from taxation are
distributed may influence attitudes towards the merit and legitimacy of taxation itself, or what the literature has sometimes termed “tax morale.”12 For example, the municipal lottery in Montevideo—which
distributes benefits to good taxpayers according to a randomized (and equitable) criterion—may affect
beliefs about the equity and fairness of the tax system, or even broader attitudes towards the government.
If such benefits are included in expressions (1)-(2), then the condition in (3) is substantially altered.
For instance, letting b be the perceived benefit of paying taxes in terms of the social return or legitimacy
of taxation, the expected utility of full nonpayment is thus z − pc − b, while the expected utility of paying
10
See Andreoni, Erard, and Feinstein 1998: 844-47.
Sandmo (2004) reviews efforts to incorporate socially conscious behavior in the tax literature; Allingham and Sandmo (1972)
consider social stigma associated with non-payment.
12
See Torgler 2002.
11
11
taxes is b + (1/10, 000)z − z. Now, tax payment occurs whenever
pc >
19, 999
z − 2b,
10, 000
(4)
which is satisfied for a greater range of parameter values than is equation (3).
Yet, is there really such an expressive benefit to paying taxes—and can the reward of a year free
of tax payment, or manipulation of the expressive benefits associated with tax payment, induce greater
tax compliance? Ultimately, this an empirical question, which our design and unusual source of data will
allow us to study. Note that the lottery induces a direct material benefit to paying taxes—and it might
also shape respondents’ view of the equity or fairness of the tax system, thereby generating a greater
expressive value to tax payment. Our research will therefore add to a small number of field experiments
that manipulate parameters that may affect compliance behavior in a real-world setting.13
In our study, the probability of detection for tax evasion takes a particular form. Here we study
the impact of positive and negative incentives to pay four kinds of municipal taxes, for which asset values
are typically known to the tax authority (for instance, through property valuation), giving taxpayers less
scope to “hide” the real value of taxes owed, relative to, say, income taxes. However, non-payment does
not result in a certainty of fines, for a number of reasons. First, there have been a number of amnesties
for bad taxpayers (e.g after the financial crisis in 2002, and in 2013), which must generate the perception
of some probability of future non-enforcement of tax violations. Second, the municipality has sometimes
renegotiated tax debts individually with taxpayers, especially large taxpayers, offering longer payment
periods and reducing principal or interest owed. Finally, even though the municipality in principle knows
asset values, non-payment may not be detected; there is an audit unit14 that produces a list of bad taxpayers;
according to one of our interviewees (an authority in the municipality), the unit is sometimes directed to
look for delinquent taxpayers in certain zones, or with certain property values, but sometimes “there is
13
Slemrod, Blumenthal, and Christian (2001) used a randomized controlled field experiment in Minnesota, in which a treatment
group of taxpayers received a letter stating that their state and federal return would be closely examined. These researchers
used 2 years of income return data to study effects on reported income, deductions, and tax liability and found that the audit
treatment increased reported income, especially among higher-income taxpayers.
14
The “Sistema de Gestión de Contribuyentes, which employs around 50 people
12
nobody looking” for particular categories of tax evasion. In total, the municipality has estimated to us
that non-payment rates are around 15 to 20 percent, though this varies by the type of tax.15 Thus, it may
make sense to think about the standard expected utility model for tax incentives, in which the benefit of
evasion is weighed against the cost of detection multiplied by its probability; what we need to investigate
is how this calculus is altered by a positive benefit to paying taxes, and our study gives us a rare empirical
opportunity to investigate this topic.
3
Empirical Strategy
We use the lottery and our informational interventions, together with administrative and survey data, to
estimate effects of distinct treatments for different types of taxpayers. The full design, including sample
sizes, is depicted in Table 3.1; the text of informational treatments appears in subsection 3.2. In our study,
we will use both administrative data (tax payment records) provided by the municipality and household
survey data (both described further in Section 4) to study the effects of the lottery, as well as the effects
of our informational interventions. Our household survey will be administered to a random sample of
individuals for whom we have administrative/tax payment data.16
3.1
A natural experiment: estimating effects of the tax holiday lottery
The tax holiday lottery provides a natural experiment that allows ready estimation of the impact of winning
the lottery among eligible taxpayers, and also allows study of several other interesting causal quantities.
To select taxpayers for holidays, the municipal government uses the results of Uruguay’s National Lottery,
which posts online five random digits that indicate winning lottery numbers. Taxpayers whose four-digit
IDs (current account numbers) correspond to the final four digits of the winning National Lottery numbers
are selected as the provisional winners of tax holidays.17 The municipality screens provisional winners to
15
We will estimate this rate using our random samples of tax payment data.
We take a sample due to the greater cost involved in data collection for the household surveys.
17
The randomization occurs through the selection from balls from an urn, as described in Spanish at http://www.loteria.gub.uy/Juego Loteria.php.
For an example of posted lottery results, see
Winning taxpayer numbers are posted at
http://www.loteria.gub.uy/ver resultados.php?vdia=21vmes=6vano=2013.
16
13
identify those whose tax accounts are paid up in the previous fiscal year, and then sends a letter to these
eligible winners (“good” taxpayers) indicating that they should register for a year free of tax payments.
For any good taxpayer, the probability of winning any given rebate lottery is 1/10,000.18 However, some
winners are not good taxpayers or physical persons or do not register. The municipality grants such
holidays six times a year (February, April, June, August, October, and December), depending on the tax.19
According to municipal authorities, the typical taxpayer in Montevideo pays around 9,000 Uruguayan
pesos in annual municipal taxes on average, or around US$400, with an average amount owed for back
taxes of perhaps 1,800 pesos (US$80). We will also be able to use our random sample of taxpayer data
to estimate these quantities. However, taxes are substantially higher for many taxpayers. In particular,
among property owners the average annual value of property taxes is over US$1,000 (24,000 Uruguayan
pesos). The value of winning a year free of tax payments therefore varies by the tax, but for property taxes
this value is quite substantial—even if the expected value is vanishingly close to zero, given the probability
of winning any lottery.20 Given that people often take gambles with negative expected values (state-run
lotteries are large money makers, after all), we think it is plausible that even a lottery with a low expected
value but a non-trivial payoff can shape tax-paying behavior.
Measuring impact among “good” taxpayers. The design of the tax holiday program allows us
to estimate the effects of winning the lottery, among “good” taxpayers. In particular, we will use a timeseries panel of administrative data to assess the effects of winning the lottery on subsequent tax payments,
comparing the payment history at t + 1, t + 2, t + 3 . . . of lottery winners in year t to a control group of
non-winners eligible to win in year t. We can assess persistence of effects by varying the number of tax
years subsequent to t.
Here, the treatment group includes all eligible lottery winners since 2004 (approximate N=7,200).
These lottery winners were “good taxpayers” by virtue of being current on their accounts over the previous
year, at the time the particular lottery that they won was conducted. The municipality will provide us
http://www.montevideo.gub.uy/sorteosBP/pages/sorteosBuenosPagadores.xhtml.
This is because each taxpayer current account ID is linked to a single final four-digit number. There are 10,000 unique
four-digit numbers and each has equal probability of selection.
19
The National Lottery frequently posts lottery results, and a different lottery result is used to select winners of rebates for each
type of tax.
20
E.g., for property taxes, US$1, 000/10, 000 = $0.1, or ten US cents.
18
14
with tax payment records for this group of winners. For the control group, we use a random sample of
eligible non-winners in each year (total N=2,000), for whom the municipality will also provide us with tax
payment data.21 Note that the set of taxpayers that comprises the appropriate control group changes for
every lottery, because a different set of taxpayers is eligible to win (as some taxpayers become current on
their payments over time, while others become delinquent). It would potentially be misleading to compare
eligible winners to a control group that includes both eligible and non-eligible taxpayers; while the former
group includes only “good taxpayers” eligible to win a particular tax lottery at time i, the latter might
include a mix of “good” and “bad” taxpayers. This could lead to bias in our estimator of the impact of
winning the lottery among good taxpayers. Thus, it is critical to construct the control group by sampling
losers who were eligible for a particular tax lottery at a particular time t.
To construct the control group properly, we will therefore sample taxpayers who were eligible to
win lottery X at time t, but who did not do so. We will do this for each lottery since 2004 to arrive at a
total sample of losers of size N=2,000.22 In addition, we will use a household survey to gather additional
information on attitudes and beliefs for a random sub-sample of eligible winners and losers (see Table
3.1). We will also estimate spillover effects by comparing average payments of the neighbors of winners
and non-winners.
3.2
Informational interventions
While our natural experiment identifies the effect of winning the lottery among eligible taxpayers, the
overall impact of the program is likely larger, because the existence of the lottery may encourage taxpayers
to become eligible (to become “good taxpayers”) in the first place. Moreover, some lottery losers probably
know about the existence of the lottery anyway, diluting the impact of winning the lottery (since one
channel through which the lottery may have an effect is informational, given that not every taxpayer
knows of the existence of the lottery). For purposes of assessing the marginal impact of the benefit, we
would also like to estimate the impact of the lottery on future tax payment behavior of good taxpayers who
21
22
See the first row, first two columns of Table 3.1.
Though administrative data will be provided to us free of charge by the municipality, recovering specific taxpayer records is
costly for the bureaucracy; thus, we cannot use a census of eligible losers in the control group.
15
learn about the lottery’s existence by winning a tax holiday. Finally, note that the lottery by itself does not
allow us to parse explanations for any effect we may find. For instance, does winning the lottery shape tax
payment behavior through an informational effect (by informing taxpayers of the existence of the lottery);
through an income effect (by making them temporarily richer, which might have unpredictable effects on
tax compliance); or by shaping their views of taxation, for instance, their beliefs about the equity and
fairness of the tax system.
To address these important limitations, we use a set of cross-cutting interventions that inform some
taxpayers about the existence of the lottery. The informational messages will appear on tax bills mailed
to households by the municipal government. We also use these informational interventions to compare
the effects of positive and negative incentives to pay taxes, by using one treatment condition in which
taxpayers are informed/reminded about sanctions for failure to pay taxes (see subsection 3.2.3). Finally,
we introduce additional, richer messages about the existence of benefits and punishment that emphasize
the social rather than individual rewards or sanctions for compliance or non-compliance. This variation
in the content of the messages helps us pin down mechanisms through which the lottery may affect tax
compliance behavior, and it also helps us answer a basic policy question: If Montevideo—or another
municipal government—were to use an informational campaign to tell citizens about the existence of the
rebate lottery, what sort of interventions would be most effective in boosting tax payments?
3.2.1
Text of messages on tax bills
We introduce four distinct messages as informational interventions, along with a control group that does
not receive any informational treatment. The following approximate language will appear on municipal
tax bills, for each of the four treatment conditions (we label each condition in CAPS, according to the
rows of Table 3.1); subjects will receive one of these messages according to their assigned condition. The
text of the four treatment conditions is as follows:
1. EXISTENCE OF LOTTERY/INDIVIDUAL BENEFIT:
“We remind you that payment YYYY of the XXXX tax is due this month. If you are upto-date with payment of the XXXX tax, you will automatically participate in the usual
16
lottery of the municipal government (IMM) that benefits good taxpayers with a year free
of payment of the tax. Each year, the IMM awards ZZZZ in benefits to good taxpayers
who win the lottery. You can be the next one!”
(Spanish text: Le recordamos que este mes vence la cuota YYYY del impuesto XXXX.
Si usted está al dı́a con el pago de XXXX participará automáticamente del sorteo habitual de la IMM que beneficia a buenos pagadores con un año de exoneración del pago del
tributo. Cada año, la IMM otorga ZZZZ en beneficios a buenos pagadores que ganan el
sorteo. ¡Usted puede ser el próximo!)
2. EXISTENCE OF FINES/INDIVIDUAL PUNISHMENT:
“We remind you that payment YYYY of the XXXX tax is due this month. If you are
not up-to-date with payment of the XXXX tax, you may be subject to fines and interest
on the debt. Each year, Montevideans pay ZZZZ in interest payments and punishments.
Pay on time and avoid fines and charges!”
(Spanish text: Le recordamos que este mes vence la cuota YYY del impuesto XXXXX.
Si usted no está al dı́a con el pago de XXXX, pueda quedar sujeto al pago de multas e
intereses a la deuda. Cada año, la IMM cobra XXXX en intereses y castigos. ¡Pague en
fecha, evite multas y recargos!
3. SOCIAL REWARD:
“We remind you that payment YYYY of the XXXX tax is due this month. If you are upto-date with payment of the XXXX tax, you will automatically participate in the usual
lottery of the municipal government (IMM) that benefits good taxpayers with a year free
17
of payment of the tax. Each year, the IMM awards ZZZZ in benefits to good taxpayers
who win the lottery. The IMM has conducted this lottery for the past ten years to
recognize good taxpayers for their contribution to the construction of a city that is
more just and better for all.”
(Spanish text: Le recordamos que este mes vence la cuota YYYY del impuesto XXXX.
Si usted está al dı́a con el pago de XXXX participará automáticamente del sorteo habitual de la IMM que beneficia a buenos pagadores con un año de exoneración del pago del
tributo. Cada año, la IMM otorga ZZZZ en beneficios a los buenos pagadores que ganan
el sorteo. La IMM realiza este sorteo hace diez años para reconocer a los buenos pagadores por su contribución a la construcción de una ciudad más justa y mejor para
todos.
4. SOCIAL PUNISHMENT:
“We remind you that payment YYYY of the XXXX tax is due this month. If you are
not up-to-date with payment of the XXXX tax, you may be subject to fines and interest
on the debt. Each year, Montevideans pay ZZZZ in interest payments and punishments..
Fines and charges are a punishment to those who evade the payment of taxes and
do not contribute to the construction of a city that is more just and better for all.”
(Spanish text: Le recordamos que este mes vence la cuota YYYY del impuesto XXXX.
Si usted no está al dı́a con el pago de XXXX, pueda estar sujeto al pago de multas y intereses a la deuda. Cada año, la IMM cobra XXXX en intereses y castigos. Las multas y
recargos son un castigo a aquellos que evaden el pago de impuestos y no contribuyen
a la construcción de una ciudad más justa y mejor para todos.)
18
Since we seek to equate the (expected) value of individual benefits and punishments across the treatment
conditions, to the extent possible, we use the same value ZZZZ across the treatment conditions, which
indicates benefits or fines according to the condition.23 Here, YYYY and XXXX are varied by the
municipality, according to the tax bill that is sent out to individual taxpayers.
Note that we do not include a pure placebo group that is distinct from the control group (e.g., in which
taxpayers are reminded of the tax due but not stimulated to think about benefits or punishments). This is
because our control group receives a reminder about taxes due—in the form of a tax bill—which a
fortiori reminds taxpayers of taxes due.
3.2.2
Parsing informational effects
These crosscutting informational interventions will allow us to estimate several different causal quantities,
beyond the simple impact of winning the lottery among eligible taxpayers. In particular:
Measuring impact among “bad” taxpayers. As noted, a limitation of the government’s program, from
an inferential point of view, is that the lottery is restricted to good taxpayers. To assess the broader impact
of the program—e.g., the effect of giving bad taxpayers greater incentives on the margin to pay their taxes
and thus gain eligibility for the lottery—we will use our informational interventions, this time focusing
on a random sample of “bad” (ineligible) taxpayers.24 Here, the difference between bad taxpayers
randomly assigned to receive messages about the existence of the lottery and a control group of bad
taxpayers estimates the intent-to-treat effect of information (first two rows, third column of Table 3.1).
In addition, an instrumental-variables estimator, which we construct using our survey data, gives an
estimate of the effect of learning about the lottery. The denominator of the estimator is the proportion of
uninformed bad taxpayers, which we estimate pooling survey data from bad taxpayers who are not
informed about the lottery; see Section 5 for details.25 This quantity is important for policy, because it
23
We believe we can do this without deception, by varying the types of taxes to which we refer; however, we are fine-tuning
this aspect of the text in conjunction with the municipality.
24
The municipality will help us identify these taxpayers from administrative records.
25
Also, Table 3.1, column 3, rows 1 and 3; N=1,500.
19
identifies the marginal impact for a population that Montevideo (or another municipality) may wish to
target if scaling up the tax holiday program: bad taxpayers who would learn about the tax holiday from
an informational campaign.
Informational vs. income effects. Winning the lottery provides a temporary income boost; yet, it may
also affect future tax compliance by shaping knowledge of the lottery, perceptions of the likelihood of
future tax holidays, or broader attitudes towards the fairness and social benefits of taxes.26 We distinguish
between these possibilities by using our crosscutting informational interventions, which take advantage
of the fact that the lottery is poorly publicized. Comparison to a control group of non-winners. Note that
comparing “good” (eligible) non-winners who receive a message on their tax bill informing them of the
existence of the lottery to a control group of eligible non-winners estimates a pure informational effect
(second column, first two rows of Table 3.1). We can compare this to the effect of winning the lottery
(first row, first and second column) to assess the effect of winning the rebate, net of informational effects.
Again, an instrumental-variables estimator gives an estimate of the effect of learning about the lottery for
uninformed good taxpayers influenced by our informational treatment. (Here, the denominator of the
Wald estimator is the proportion of uninformed good taxpayers, which we estimate pooling survey data
from column 2, rows 1 and 3, of Table 3.1; N=1,500).
Measuring effects for “would-be winners.” The municipality will mail a special message on the tax bill
to a sub-sample of bad taxpayers, whose tax ID numbers would have made them lottery winners in the
past—had their tax payments only been up-do-date (this is the sub-sample denoted by an asterisk in row
2, column 3 of Table 3.1). The messages from the municipality will inform them of this fact. Note that
the municipality never contacts delinquent taxpayers after lotteries (as it screens payment records before
contacting winners), so such taxpayers will typically have no idea that they would have won, had they
been good taxpayers. We are interested in comparing the effect of this message to informational effects
for other bad taxpayers. If there is any any detectable difference, we expect it to come from the especially
powerful priming (perhaps of “regret”) provided by this treatment.
26
Winning lotteries can also engender self-reinforcing beliefs about individual merit, which might have broader political implications. See e.g. Di Tella et al. (2007).
20
We describe these hypotheses and operationalization of tests further in Section 5.
3.2.3
Positive vs. negative incentives for tax compliance
How do the effects of the positive incentives provided by the lottery compare to the effects of negative
incentives/sanctions? In developing countries where tax enforcement is routinely difficult, the effects of
information about sanctions should provide a useful benchmark for the effects of knowledge of rewards.
One previous informational experiment found that emphasizing fines and other legal consequences of nonpayment increased tax compliance in Argentine municipalities by more than 4 percentage points.27 Our
design replicates the spirit of this intervention (our Existence of Fines/Individual Treatment condition is
similar to a condition in Castro and Scartascini 2013) and compares it to the effect of information about
the rebate lottery, among both good and bad taxpayers (first and third rows, final columns of Table 3.1).28
As with our other informational interventions, we will estimate these effects by tracking compliance using administrative data on payments (subsequent to the mailing of flyers) and also through survey data on attitudes and perceptions for a subset of citizens. In a developing-country context in which
tax enforcement is routinely difficult, the effects of information about sanctions should provide a useful
benchmark for comparing the effects of information about rewards. Since not all taxpayers may be aware
of sanctions and fines for non-payment, here too we can also construct instrumental-variables estimates in
which the denominator is the difference in the proportion of the treatment and control groups that is aware
of sanctions (here, the treatment group is those taxpayers exposed to the Existence of Fines/Individual
Treatment condition).
27
28
Castro and Scartascini (2013).
The text of the relevant treatment from Casto and Scartascini (2013), which was administered in the Argentine municipality
of Junı́n, is as follows: “Did you know that if you do not pay your [municipal tax] on time, for a debt of, for example,
1,000 pesos, you will have to pay 268 additional pesos at the end of the year, and the municipality can sanction you with
administrative and even judicial measures? (In Spanish: ¿Sabı́a Usted que si no paga a tiempo el CVP, para una deuda de, por
ejemplo, 1.000 pesos deberá pagar 268 pesos adicionales a fin de año, y el municipio puede llegar a intimarlo administrativa
y hasta judicialmente?”
21
3.3
Treatment assignment and design matrix
We will assign the 24,200 taxpayer records that we obtain to the treatment conditions shown in Table 3.1
to one of the informational conditions. Each individual taxpayer for whom we obtain administrative data
will have equal probability of assignment to each of the conditions. Treatment assignment is blocked by
type of tax (recall that a different lottery is used for each type of tax, and there are different numbers of
payers of each tax, so the probability of assignment vary by block). Our analysis will account for this
blocked assignment (e.g. through inverse probability weighting). Table 3.1 depicts the complete design.
The sample sizes are justified in the next section, as are data collection procedures.
Table 3.1: Treatment Conditions and Sample Sizes
Treatment Conditions
(Randomized Tax Holiday Lottery)
Treatment Conditions
Lottery winners
Lottery non-winners
(Information Experiment)
(Good taxpayers)
(Good taxpayers)
Control
Admin. Data, N=7,200 Admin. Data, N=2,000
(+ Surveys, N=1,000)
(+ Surveys, N=1,000)
Existence of Lottery/
N.A.
Admin. Data, N=1,500)
Individual Benefit
(+ Surveys, N=1,000)
Existence of Fines/
N.A.
Admin. Data, N=1,500
Individual Deterrent
(+ Surveys, N=500)
Social Reward**
N.A.
N=1,000 (Admin. Only)
Social Punishment**
N.A.
N=1,000 (Admin. Only)
TOTAL N
Admin. Data, N=7,200 Admin. Data, N=8,000
(+ Surveys, N=1,000)
(+ Surveys, N=2,000)
Sample of ineligibles
(Bad taxpayers)
Admin. Data, N=2,000
(+ Surveys, N=1,000)
Admin. Data, N=2,500*
(+ Surveys, N=1,000)*
Admin. Data, N=1,500
(+ Surveys, N=1,000)
N=1,000 (Admin. Only)
N=1,000 (Admin. Only)
Admin. Data, N=9,000
(+ Surveys, N=2,500)
Total N=24,200 (administrative data); we will collect survey data for a stratified random sub-subsample
of N=5,500. * This group includes an additional sub-sample of bad taxpayers who would have won
the lottery based on their account IDs, had their accounts been up to date (N=1,000 for administrative
data, with survey data for N=500); messages will inform them of this. ** The effects of the italicized
informational/framing treatments will be measured through administrative data only. See subsection 3.2.1
for text of informational treatments.
The municipality will then mail tax bills to the addresses given in the taxpayer records; each bill
will print the message associated with one of the informational treatments (or no message, in the control
group), according to treatment assignment. The fact that the messages are printed on tax bills boosts the
experimental realism and policy relevance of the intervention. Bills will be mailed a few weeks before the
22
relevant tax deadline, for each type of tax. Behavioral outcomes will be assessed using administrative data
on tax compliance; effects on attitudes and perceptions will be assessed using household survey data (see
Section 4).
4
4.1
Data collection
Administrative tax payment data
We have permission from the municipal bureaucracy to access selected taxpayer records from 2004 to
2013. The records include taxpayers’ current account (e.g. amount paid per year for 2004 to 2013),
appraised value of property/vehicle, and tax rate; billing address; and often taxpayers telephone or email.
This will allow us to define various measures of tax compliance and also locate treatment and control
households for our surveys. Our study period will continue until September 2015 to allow us to assess the
persistence of effects of our informational interventions, using administrative records.
In more detail, following the design in Table 3.1, we will obtain taxpayer records for all eligible
lottery winners since 2004 (circa 7,200 taxpayers), a random sample of eligible (“good”) taxpayers who
have not won the lottery (N=8,000), and a random sample of ineligible (“bad”) taxpayers (N=9,000).
The municipal government will identify records of all the lottery winners; for lottery losers and ineligible
taxpayers, we will randomly generate four digit numbers, screen out any numbers that coincide with
lottery winners, and supply these numbers to the municipality. For lottery losers and bad taxpayers, we
distribute the sample across the four types of municipal taxes in proportion to their distribution among the
7,200 lottery winners (i.e., if 4,000 of the lottery winners come from payers of the property tax, we will
draw (4000/7,200)*100 or 55.5% of the sample of lottery losers and ineligible taxpayers from property
tax records.
For eligible lottery non-winners, a further detail is crucial, as noted previously: the proper control
group consists of taxpayers who were eligible to win a given lottery of type X at time t, but did not do
so. In effect, there are many mini-experiments in our study, six for each year-tax lottery combination.
Suppose that there are 50 winners of a given lottery, say, property tax lottery in March 2010. Given this
23
treatment group, we will then sample 50 eligible non-winners—e.g.., property taxpayers who were “good
taxpayers” (current over the previous year) in March 2010—by supplying the municipality with random
four-digit numbers sufficient to generate a proper control sample of this size. Note that some random four
digit numbers we draw may not generate valid taxpayer records, or a sufficient number of them; thus, we
will iterate until we have obtained the desired sample size. We will also randomly draw a small number of
alternate taxpayer records that we can use to replace individuals whom our survey firm is unable to contact
or who refuse to participate in surveys (see next section); however, in analysis we will be able to separate
out and identify these alternates, as needed.
The sample size for administrative data collection balances our desire for more data against the cost
in time and effort to the municipality, and our need to secure the municipality’s cooperation in providing
data. (Supplying data is not costless for the municipality, as will involve manual extraction of records from
municipal databases using the four-digit IDs we generate, and the municipality will only grant access
to a sample of the data). As per our power calculations in subsection 4.4, we believe the sample of
administrative records is adequate for the study.
4.2
Household sample and survey design
We will take a random sample of taxpayers for whom we obtain administrative records, including 1,000
lottery winners, 2,000 eligible lottery losers, and 2,500 ineligible taxpayers (Table 3.1). The administrative
data list taxpayer names, addresses and in some cases phone numbers, and our survey firm will use those
to track down sampled individuals. In some cases we may allow limited substitutions from the randomly
drawn list of alternates, in cases where sampled individuals cannot be found or refuse to participate (see
previous section). We have received quotes from several survey firms in Uruguay (e.g. EQUIPOS MORI
and CIFRA).
The survey instrument will gather data on individual covariates and the main attitudinal dependent
variables in our analysis: for example, perceived equity and fairness of the tax system; perceptions of
the benefits provided by taxation; and a host of attitudinal (e.g., belief in individual merit) and political
(e.g. support for the incumbent Frente Amplio) variables that may be affected by winning the lottery or by
24
knowledge of the lottery. These data on beliefs and perceptions will help us assess mechanisms that may
explain any effects we estimate, as discussed in the proposal. The survey instrument will also measure
knowledge of the rebate lottery, whether respondents know anyone who has won the lottery, and related
variables, which will allow us to assess the likely effects of advertising the tax rebate program more widely.
See Section 5 for further details.
4.3
Other outcomes: call data from flyer recipients
We may set up a phone number with voice mail to register any reactions from taxpayers to the messages
on tax bills; we will distribute this number with the flyers. Any complaints will be passed on to the
municipality but may also provide interesting outcome data in their own right.
4.4
Power calculations and sample sizes
How big should our samples of administrative data and households for our survey be? Here we justify with
power calculations the sample sizes in Table 3.1. We take as a benchmark effect size the informational
experiment of Castro and Scartascini (2013), who estimate effects of informational treatments on tax
compliance of over 4 percentage points using negative incentives for compliance (reminding taxpayers of
punishment and fines for non-compliance). However, we also calculate the probability of rejecting the
null hypothesis of no effect, given various true effect sizes. This true effect could be, e.g., the difference
in subsequent tax compliance rates for after winning the lottery, compared to the counterfactual of not
having won the lottery. There are N units with nT units assigned to treatment and nC = N − nT to control;
for the power calculations, we assume equal numbers assigned to treatment and control (nC = nT ), as in
some of the pairwise comparisons in Table 3.1 and as will often be the case when we pool across different
treatment conditions in our informational experiment.
We assume a conservative estimate of average tax compliance of around 70% (though the municipality has said non-compliance is around 15-20%, so 80% is another alternative). Thus, we assume that
the variance of this binary outcome is 0.7 × (1 − 0.7), pooling across treatment and control groups. The
25
standard error for the difference of tax compliance rates across treatment and control groups is29
r
or, using nT = nC = N2 ,
0.7 × 0.3 0.7 × 0.3
+
,
nT
nC
(5)
√
2 0.7 × 0.3
.
SE =
√
N
(6)
For each effect size, we calculate power under a two-tailed test as
1 − Φ(2 −
effect
),
SE
(7)
where Φ is the normal cumulative distribution function, SE is given by equation (6), and
effect is the true effect size.30 Equation (7) gives the approximate area above the normal curve centered
over effect that is more than two standard errors above 0, the effect size under the null hypothesis.31 For
a one-tailed test, we use 1.65 in place of 2 in equation 7; a one-tailed test is more appropriate for some of
our unidirectional hypotheses (see Section 5).
For this binary outcome, we assume true effects of 4, 6, 8, and 10 percentage points, e.g.,
effect ∈ {0.04, 0.06, 0.08, 0.10} (Figures 1 and 2).32 In each figure, the vertical line shows the study
size, pooled across treatment and control groups, that is needed for 80% power given each effect size.
For N = 2, 000, we have slightly more than 80% power given a true effect size of 6 percentage points,
using a two-tailed test; for a one-tailed test, we have 80% power against a true effect size of 5 percentage
points (N = 2, 000). With a one-tailed test, we also have 80% power for an effect size of 6 percentage
points when N = 1, 500. These calculations suggest our power to measure effects with binary outcomes
using our survey data. However, to measure the binary outcome of tax compliance, we will use cheaper
29
We use the “conservative” formula for the standard error in randomized experiments, which is the same as for the difference
of proportions of two independent samples; for formal justification, see Appendix notes 31, 33 of Freedman, Pisani, and
Purves (2007). See also Dunning (2012, Chapter 6) or Gerber and Green (2012).
30
We switch the signs in (7) to give the area greater than two standard errors above zero.
31
To be These effect sizes are similar to those estimated in municipal tax compliance experiment of Castro et al. in Argentina.,
here we use 2 in place of 1.96, though we can rely on the central limit theorems and use normal approximations for most
hypothesis tests; with smaller n, one might want to use the t-distribution or permutation tests.
32
Thus, we take benchmark effect size of 4 percentage points as the most demanding case from the point of view of power.
26
administrative data and thus a larger N, so our power will be substantially greater. For the comparison of
lottery winners and lottery losers using administrative data (pooled N = 9, 200), our power is about 80%
against a 2.7 percentage point difference in tax compliance.
Power is also greater with graded measures rather than binary outcomes. For example, our household survey will measure attitudes towards the tax system, often using scales instead of binary outcomes
(e.g., degree of agreement with statements about the fairness of the tax system). In Figures 3 and 4,
we measure effect sizes in relation to the unknown standard deviation of this outcome scale. Thus, for
N = 2, 000, we will have power of just over 80% against a true effect size of 0.13 standard deviations
(two-tailed test), and for N = 1, 500, we will have 80% power against a true effect size of 0.15 standard
deviations.
What is our power to estimate population parameters in each cell of Table 3.1? One important
role of the household survey—besides its primary use in generating data on perceptions and beliefs and
thereby allowing us to assess mechanisms that might explain tax compliance effects—is to allow us to
estimate the proportion of taxpayers in Montevideo who are uninformed about the existence of the lottery.
It is critical that we estimate this proportion precisely, as this estimated proportion is the denominator in
our instrumental-variables analysis. For this purpose, we have N = 1, 500 among good taxpayers who are
lottery losers and N = 1, 500 among “bad” taxpayers who are ineligible for the lottery. (This uses the units
in the control group in the experiment and the units assigned to the “Existence of Fines” condition, whom
we do not inform about the existence of the lottery—see Table 3.1). If 50% of good taxpayers who have
never won the lottery are unaware of its existence, the standard error for our estimate of the population
percentage is 3.3%. When pooling across good and bad taxpayers for N = 3, 000, the standard error is
1.7%.
Sample size justification. These power calculations justify our sample size for the household
survey, depicted in Table 3.1. Our power is about 80% against effect sizes for tax compliance comparable
to those estimated in previous research. Our sample size gives us similar power against movements of
around 0.15 standard deviations in attitudinal dependent variables measured as scales. Finally, our sample
of households who have not won the lottery allows us to estimate the proportion of taxpayers who are
27
uninformed about the lottery with fairly good precision; these estimates are important for assessing overall
program impact as well as the likely effects of more effectively promoting knowledge of the tax rebate
lottery.
5
Analysis plan
In this section, we lay out the main hypotheses and statistical tests to be conducted, describing the comparisons we will make as well as the source of outcome variables. We place emphasis on counting the
number of tests, so as later to allow for meaningful adjustment for multiple statistical comparisons. Note
however that this is a draft, subject to revision as we refine our design and outcome measures; we’ll post
a version at a study registry before data collection and analysis.
5.1
Hypotheses and tests
MAIN EFFECTS OF THE TAX HOLIDAY LOTTERY
Main effects for “good taxpayers”—Compliance: Against the null hypothesis that winning the
lottery has no effect on tax compliance, we test the alternative of an effect, using a two-tailed test and
(given the sample sizes) normal approximations for the hypotheses tests. A two-tailed test is appropriate
since, while the most likely alternative is that winning the lottery increases future compliance, it could
in principle decrease it—for example, if there are income effects, such that winning the lottery makes
taxpayers less sensitive to the possibility of future fines for non-payment.33 However, we expect that by
informing taxpayers of the existence of the lottery and/or by shaping their views of the equity and fairness
of the tax system, a positive effect on compliance is the more likely of the two alternative hypotheses.
Outcome measure: Tax compliance in year t + x, after winning a tax lottery in year t (for winners) or after
the year of eligibility for winning a tax lottery (for non-winners). We will vary x to assess the persistence
of effects; thus, x = 1, x = 3 (for taxpayers who won lotteries before 2011), and x = 5 (for taxpayers who
33
It could also make respondents believe—falsely—that their probability of winning future lotteries is diminished, if they have
already won.
28
won lotteries before 2009). The outcome variable is 1 if the taxpayer is a good (eligible for the lottery)
taxpayer at time t = x and 0 otherwise. We will also create a summary tax compliance score: the number
of years in which a taxpayer has been a “good taxpayer” since t, divided by the number of years since t.
So that is four tests of the main effect on compliance among “good taxpayers.”
Main effects for “good taxpayers”—Attitudes: We conduct similar two-tailed tests as for compliance but now with several survey items measuring attitudes towards taxation. Outcome variables: 1.
“On a scale of 1 to 7, where 7 means ‘very good’ and 1 means ‘very bad’, where would you rate the municipal services in Montevideo?” 2. “On a scale of 1 to 7, where 7 means ‘totally agree’ and 1 means ‘totally
disagree’, where would you put your agreement with the following sentence: the interests and needs of
Montevideans are taken into account by City Hall?” 3. “On a scale of 1 to 7, where 7 means ‘totally agree’
and 1 means ‘totally disagree’, where would you put your agreement with the following sentence: the tax
system of the municipal government is fair and equitable.” 4. “On a scale of 1 to 7, where 7 means ‘totally
agree’ and 1 means ‘totally disagree’, where would you put your agreement with the following sentence:
The performance in office of the current municipal government is very good.” 5. “On a scale of 1 to 7,
where 7 means ‘totally support’ and 1 means ‘do not support at all’, where would your degree of support
for the political party of the municipal government, the Frente Amplio.”
This makes five tests of the main effect on political attitudes among “good taxpayers.”
MAIN EFFECTS OF INFORMATIONAL TREATMENTS
Main effects for “bad taxpayers”–knowledge of the lottery: Here we compare “bad” (ineligible)
taxpayers who are informed about the existence of the lottery/the individual benefit to the control group
(final column, first and third rows of Table 3.1). As we will only have taxpayer compliance data for the
year following the administration of our treatments (unless we do later follow-up study), here we gather
outcome data at only t + 1, making for one two-tailed test of the null hypothesis of no effect for the 0-1 tax
compliance variable. We will also test effects on the five attitudinal variables measured for good taxpayers,
which makes six tests. Instrumental-variables analysis: For each of these outcome variables, we also want
to estimate the impact on bad taxpayers who learned about the lottery from informational treatment (in the
29
language of the treatment effects literature, these taxpayers are “Compliers”). Let Y T be the sample mean
in the treatment group and let Y C be the sample mean in the control group; let X T be the proportion of
informed taxpayers in the treatment group and XC be the proportion of informed taxpayers in the control
group. Our instrumental-variables estimator is then
YT − YC
.
X T − XC
(8)
This estimates the effect of treatment for Compliers.34 The numerator is just the main (“intent-to-treat”)
effect.
Heterogeneous effects for “bad taxpayers”–knowledge of the lottery: As noted previously, the
effect of knowledge of the lottery may depend on how “in arrears” taxpayers are: some taxpayers are in
arrears a small amount, so that with a small payment they could become eligible for the lottery; some are
more seriously delinquent, such that the cost of becoming a “good taxpayer” is very high. Though it may
not be strictly rational from a material point of view for either group to become a good taxpayer—as noted,
the expected value of playing the lottery is very low—we nonetheless hypothesize that treatment effects
may be greater for less delinquent taxpayers. To operationalize this idea, we will compare the effects of
knowledge of the lottery for taxpayers whose amount owed is above average (among delinquent taxpayers)
to the effects of knowledge of the lottery for those whose amount owed is below average. We will also
compare those whose asset values are above average to those whose asset values are below average. Thus,
these hypotheses about heterogeneous effects generate four additional tests.
Main effects for “would-be winners”–knowledge of the lottery: These tests are analogous to
the tests of main effects for other bad winners, but among the subsample of bad taxpayers who would have
won the lottery based on their tax ID numbers, had their tax payments only been up-to-date. We conduct
one-tailed tests of the hypothesis that effects for this group are larger than for the main sub-sample of bad
taxpayers. Since there are six outcome variables (compliance at t + 1 plus the five attitudinal variables
tracked above), we have six additional tests.
34
In econometrics, this is sometimes called the Wald estimator—see Angrist and Pischke (2008).
30
Social rewards and social punishments: Using administrative data on tax compliance at t+1, t+3,
t+5, and our summary measure of proportion of post-treatment years spent as a good taxpayer, we will also
conduct two-tailed tests of the hypothesis of a difference in effect between the social benefit vs. individual
benefit (existence of lottery) conditions and the hypothesis of a difference across the social punishment
vs. individual punishment conditions, against the null of no effect. With four outcome variables and
comparisons for benefits and for punishments, here we have eight additional tests.
Mechanisms: Information vs. Income: Winning the tax holiday lottery is a “bundled” treatment,
in the sense that it combines information about the existence of the lottery (for taxpayers who do not know
about it) with the income that one gains from winning a year free of tax payments. To distinguish these
channels, we rely on the variation in treatment provided by our informational treatments. In particular, we
will compare the effect of winning the lottery, among good taxpayers, to the effect of being informed about
the lottery through a message on the tax bill. (Both effects are estimated by comparing treatment groups
to a pure control group, among good taxpayers). We then assess whether these effects are statistically
different from each other; if they are, we can conclude that the income effect adds explanatory power to
the information effect. For our informational treatment, we only have tax data from the following year
(t + 1), so there is just one test to be performed. (We can also compare instrumental variables estimates of
both quantities, however.
One caveat about detecting mechanisms is in order, however. Although we have designed our
informational interventions in order to parse mechanisms through variations in treatment—using what
Gerber and Green (2012) call “implicit mediation analysis”—our ability to infer channels through which
the lottery affects tax compliance behavior is necessarily limited. One real possibility is that the fact of
winning a lottery itself induces changes in attitudes or tax-compliance behavior—independent of any information about governance or any income effects that winning induces. In some sense this is analogous to
a violation of an “exclusion restriction,” for example, a restriction that the lottery does not induce changes
in attitudes or behavior through channels other than income or information. Such caveats notwithstanding,
our design may still answer the first-order question of whether a randomized benefit can shape compliance
behavior, which is the question of most policy (if not necessarily social-scientific) relevance.
31
In sum, this sub-section describes (and will describe further upon revision) the hypotheses we plan
to assess and the tests we will perform. At present, the total number of tests is on the order of 34. We
have a number of randomizations so adjustments of multiple comparisons need to take that into account.
Outcome variables are dependent, so we need an adjustment that takes this into account (such as the false
discovery rate). Future versions of this analysis plan will specify our plans for adjusting for multiple
comparisons.
5.2
Additional survey questions
We will ask several questions that we do not intend to use for hypothesis testing, for example,
• How likely is it that you will win the lottery? (But this could be an outcome variable).
• “Have you heard about the tax holidays that the Montevideo City Hall (IMM) raffles to eligible
taxpayers who are up-to-date on their taxes?” (this is post-treatment for some respondent, but we
will also use responses in the control group to estimate the incidence of knowledge in the population,
among good and bad taxpayers)
• “Do you know anyone who has won one of the tax holidays that the Montevideo City Hall (IMM)
raffles to eligible taxpayers who are up-to-date on their taxes?”
Finally, we will ask questions tapping a range of individual covariates (age, education, . . . ) (these
are post-treatment but we don’t expect they will be shaped by our interventions). We will also ask for retrospective recall about pre-treatment variables, for instance, vote behavior in past years. We are currently
developing the survey instrument and particularly welcome suggestions and feedback in this regard.
5.3
Logistics
The IMM has agreed to provide taxpayer records and has provided a letter of support in connection with
a grant proposal. However, we are still planning the logistics of obtaining the administrative data and are
also working out the procedure for mailing flyers in the informational experiment (e.g., whether flyers will
32
arrive directly with the tax bill). This poses some risk of failure if the municipality balks at the time and
effort required to supply administrative data, or at the risk of taxpayer reaction to some of the information
treatments, but we believe the can be managed successfully. We are applying for Human Subjects approval
at UC Berkeley. We believe this project poses no appreciable risk of harm to subjects.
Data will be posted upon publication of a research report or two years after the start of our informational intervention, whichever is sooner. After revision and extension, we will post this pre-analysis plan
at a public study registry before data collection, where it can receive a time stamp for registration.35 We
believe this may be among the first registered pre-analysis plans for a randomized natural experiment (the
rebate lottery). We believe this sets a useful precedent, as filing data analysis plans for natural experiments
is often feasible (design ideas typically precede data collection and analysis—that may be true in some
standard observational studies, too).
6
Conclusion: Relevance, Contribution, and Value of Research
Promoting tax compliance is critical in developing countries, where tax monitoring and enforcement is
often weak. We believe have a valuable opportunity to use a randomized policy intervention to learn
about the effects of positive as well as negative incentives for compliance. Theories of taxation and tax
compliance make claims about how individual taxpayers weigh the costs and benefits of tax evasion,
but they less frequently consider how the benefits of paying taxes may influence compliance. Empirical
studies often have little traction over this question as well, since the assignment of benefits is usually
confounded by other factors that could influence compliance. To be sure, our study focuses on a particular
kind of material benefit—the assignment of tax holidays—but these may also influence attitudes towards
the equity and fairness of the tax system, and our design will allow us to assess effects on such outcomes.
Thus, we can provide some relatively rare evidence on the effects of positive incentives on compliance.
The results may have some policy relevance as well, both for municipal authorities in Montevideo and for other governments that may wish to increase tax revenues through similar policy innovations.
35
See for instance the EGAP registry: http://e-gap.org/design-registration/registered-designs/.
33
Our informational intervention will allow us to conduct cost-benefit analysis on scaling up promotion of
knowledge of the rebate lottery. If treatment effects are non-trivial, the benefits of such policies may
well outweigh the costs to municipal authorities. The overall cost of lotteries for the government is relatively small.36 Moreover, results of the study should allow us to inform the IMM or other governments
about the most effective ways to scale up and advertise the lottery through informational interventions.
Demonstrating the benefits of such study may encourage further reform and improvement in systems of
tax administration, including not just auditing capacity but also and especially the administrative tools
needed to maintain the lottery policy.
Of course, our results may also generate new puzzles. If we find that benefits are successful in
boosting compliance, this would raise the question of why the adoption of such incentive schemes are not
more widespread. That lotteries often seem politically popular makes the question more intriguing. We
view the empirical assessment of this lottery’s effects as a first step in understanding the political economic
incentives that might prevent or enable the adoption of more effective tax collection programs, and thus
the building of greater state capacity in developing-country settings.
36
For example, with on average 50 winners of every property tax lottery and an average payout of 24,000 Uruguayan pesos, the
cost per lottery is on the order of 1.2 million pesos (around US$50,000).
34
7
References
Acemoglu, Daron and James A. Robinson. 2001. “A Theory of Political Transitions.” American Economic
Review 91: 938-63.
Acemoglu, Daron and James A. Robinson. 2006. Economic Origins of Democracy and Dictatorship.
Cambridge University Press.
Allingham, Michael G. and Agnar Sandmo. 1972. “Income Tax Evasion: A Theoretical Analysis.” Journal
of Public Economics 1: 323-38.
Alm, James, Betty R. Jackson, and Michael McKee. 1992. “Deterrence and Beyond: Toward a Kinder,
Gentler IRS.” In Joel Slemrod, ed., “Why People Pay Taxes: Tax Compliance and Enforcement.” Ann
Arbor: U. of Michigan Press, pp. 311-29.
Andreoni, James, Brian Erard, Jonathan Feinstein. 1998. “Tax Compliance.” Journal of Economic Literature 36 (2): 818-860.
Angrist, Joshua D. and Jörn-Steffen Pischke. 2008. Mostly Harmless Econometrics. Princeton University
Press.
Bates, Robert and Da-Hsiang Lien. 1985. “A Note on Taxation , Development, and Representative Government.” Politics and Society 14 (53): 53-70.
Beck, Paul J., Jon S. Davis, and Woon-Oh Jung. 1991. “Experimental Evidence on Taxpayer Reporting
under Uncertainty.” The Accounting Review 66 (3): 535-58.
Becker, Winfried, Heinz Jurgen Buchner, and Simon Sleeking. 1987. “The Impact of Public Transfer
Expenditures on Tax Evasion: An Experimental Approach.” Journal of Public Economics 34 (2): 24352.
Besley, Timothy, and Torsten Persson. 2009. “The Origins of State Capacity: Property Rights, Taxation,
and Politics.” American Economic Review 99 (4): 1218-44.
Boix, Carles. 2003. Democracy and Redistribution. Cambridge University Press.
Castro, Lucio, y Carlos Scartascini. 2013. ?Tax Compliance and Enforcement in the Pampas. Evidence
from a Field Experiment.? IDB Working Paper No. 472. Washington, DC: Inter- American Development
35
Bank.
Di Tella, Rafael, Sebastian Galiani, and Ernesto Schargrodsky, 2007, “The Formation of Beliefs: Evidence
from the Allocation of Land Titles to Squatters,” Quarterly Journal of Economics, 122: 209-41.
Dunning, Thad. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach. Cambridge
University Press.
Freedman, David, Roger Pisani, and Roger Purves, 2007, Statistics, W.W. Norton Co., 4th edition.
Friedland, Nehemiah, Shlomo Maital, and Aryen Rutenberg. “A Simulation Study of Income Tax Evasion.”
Journal of Public Economics 10 (1): 107-16.
Gerber, Alan and Donald Green. 2012. Field Experiments: Design, Analysis, Interpretation. W.W. Norton
& Co.
Levi, Margaret. 1988. Of Rule and Revenue. Berkeley: University of California Press.
Meltzer, Allan H. and Scott F. Richards 1981. “A Rational Theory of the Size of Government.” The Journal
of Political Economy 89 (5): 914-27.
Sandmo, Agnar. 2004. “The Theory of Tax Evasion: A Retrospective View.” Discussion Paper 31/04
prepared for the Nordic Workshop on Tax Policy and Public Economics, Helsinki.
Scheve, Kenneth, and David Stasavage. 2012. “Democracy, War, and Wealth: Lessons From Two Centuries
of Inheritance Taxation.” American Political Science Review 106 (1): 82-102.
Slemrod, J., M. Blumenthal, and C. Christian. 2001. “Taxpayer Response to an Increased Probability of
Audit: Evidence from a Controlled Experiment in Minnesota.” Journal of Public Economics 79: 455-83.
Tilly, Charles. 1990. Coercion, Capital and European States: D 990-1992. Oxford: Blackwell Publishers.
Torgler, Benno. 2002. “Speaking to Theorists and Searching for Facts: Tax Morale and Tax Compliance in
Experiments.” Journal of Economic Surveys 16 (5): 657-83.
36
Figure 1
Power Calculations for Different True Effect Sizes
(Two-tailed tests, Binary Outcome)
3000
5000
0 1000
Effect= 0.08
Effect= 0.1
3000
5000
0 1000
Study size
5000
0.0 0.4 0.8
Study size
Power
0 1000
3000
Study size
0.0 0.4 0.8
Power
0 1000
0.0 0.4 0.8
0.4
Power
0.8
Effect= 0.06
0.0
Power
Effect= 0.04
3000
5000
Study size
Plots show statistical power as a function of study size for different effect
sizes (binary outcome, e.g tax compliance). Effects are differences of
proportions. Vertical line shows the size required for 80% power.
37
Figure 2
Power Calculations for Different True Effect Sizes
(One-tailed tests, Binary Outcome)
5000
0 1000
3000
Study size
Effect= 0.08
Effect= 0.1
3000
5000
0 1000
Study size
5000
0.2 0.6 1.0
Study size
Power
0 1000
0.2 0.6 1.0
Power
0.6
3000
0.2 0.6 1.0
0 1000
Power
Effect= 0.06
0.2
Power
Effect= 0.04
3000
5000
Study size
Plots show statistical power as a function of study size for different effect
sizes (binary outcome, e.g tax compliance). Effects are differences of
proportions. Vertical line shows the size required for 80% power.
38
Figure 3
Power Calculations for Different True Effect Sizes
(Two-tailed tests, Graded Outcome)
5000
0 1000
3000
Study size
Effect= 0.2
Effect= 0.25
3000
5000
0 1000
Study size
5000
0.0 0.4 0.8
Study size
Power
0 1000
0.0 0.4 0.8
Power
3000
0.0 0.4 0.8
0 1000
Power
Effect= 0.15
0.0 0.4 0.8
Power
Effect= 0.1
3000
5000
Study size
Circles show statistical power as a function of study size for different effect sizes
(graded outcome, e.g attitude scales). Effect sizes are expressed in standard
deviations, e.g. 0.1 of one SD. Vertical line shows the size required for 80% power.
39
Figure 4
Power Calculations for Different True Effect Sizes
(One-tailed tests, Graded Outcome)
5000
0 1000
3000
Study size
Effect= 0.2
Effect= 0.25
3000
5000
0 1000
Study size
5000
0.0 0.4 0.8
Study size
Power
0 1000
0.0 0.4 0.8
Power
3000
0.0 0.4 0.8
0 1000
Power
Effect= 0.15
0.0 0.4 0.8
Power
Effect= 0.1
3000
5000
Study size
Circles show statistical power as a function of study size for different effect sizes
(graded outcome, e.g attitude scales). Effect sizes are expressed in standard
deviations, e.g. 0.1 of one SD. Vertical line shows the size required for 80% power.
40