THE IMPACT OF
RESTAURANT LETTER
GRADES ON TAXES
AND SALES:
EVIDENCE FROM
NEW YORK CITY
Michah Rothbart a
Amy Ellen Schwartza, Thad Calabresea , Tod
Mijanovicha, Rachel Meltzerb, Diana Silvera,
Meryle Weinsteina, and Jin Yung Baea
IESP Summer Seminar
June 24, 2014
aNew
York University
bThe New School
Potential Impacts of Public Restaurant Grades
• Public restaurant grades may
• increase consumer information at the point
of consumption decisions
• change where consumers bring business
• induce restaurants to improve hygiene and
food safety practice
• reduce foodborne illnesses
• For restaurants this may mean:
• increased investment in maintenance and
hygiene
• increased food sales for restaurants with
better food safety practices
• decreased food sales for restaurants with
poorer food safety practices
Little Evidence on Restaurant Grades’ Impacts
• Evidence on restaurant grading policies (Los Angeles)
• Jin and Leslie (2003):
• improved inspection scores
• restaurant revenues sensitive to grades
• foodborne illness hospitalizations decrease
• Simon et al. (2005):
• Foodborne illness decreases compared to statewide trend
• Public grading used in other policy areas and has economic
impacts
• Figlio and Lucas (2004):
• school report card grades affect house prices above and beyond estimated
effects of test scores
• effect has gotten smaller over time
NYC public restaurant grades began in 2010
• Prior to implementation:
• inspection scores and violations
online,
• temporary closures for uncorrected
public health hazards,
• fines assessed,
• not publicized,
• no grades
• Restaurant food safety
information may be
asymmetrical
NYC public restaurant grades began in 2010
• After implementation:
• scores, violations, and grades online,
• temporary closures for uncorrected
public health hazards,
• fines assessed,
• restaurants conspicuously post the
letter grade in their window
B
• grade based on number and severity of
inspection violations
• posted grades: “A”, “B”, “C”, “Grade
Pending”
• Inspection frequency depends on
previously earned scores
Simplified Model Of The Inspection Cycle
Initial
Inspection
Re-inspection
Adjudication
Adjudication
Fines assessed for violations at each inspection that does not lead to an A
Data
• Inspection data from the City Department of Health and Mental
Hygiene Bureau of Environmental Surveillance (DOHMH)
• 21 fiscal quarters of inspection data (10 before and 10 after)
•
•
•
•
•
EIN, address, number employees, number seats,
cuisine, venue type, service type,
date opened and date out-of-business (OOB)
inspection date, type (initial and re-inspection), score, grade,
fines assessed, and adjudication date, scores, and docket numbers
• 222,519 inspections of 40,640 restaurants
• Sales data from City Department of Finance Office of Tax Policy (DOF)
• Single-filing entities matched on EIN, de-identified, randomly assigned to
groups of ~10
• Random assignment stratified on quarters of operation
• Restaurants stay in same bin
• Summary statistics for sales, sales taxes, and DOHMH data by quarter
• Sample includes 28,264 entities in 2,504 groups (over 20 quarters)
• 70% of inspected entities included in our sample
Restaurant Descriptive Statistics
Pre Grades
Pre Grades
Post Grades
Post Grades
Borough
Cuisine offered
American
23%
23%
Manhattan
41%
40%
Chinese
10
12
Bronx
10
10
Pizza and/or Ital
11
13
Brooklyn
24
24
Other*
45
50
Queens
23
22
Missing
11
0
Staten Isl.
3
4
Num / Rest
Service type
Takeout-Lim Eat In
38%
37%
Insp/Yr
1.3
2.6
Wait service
17
20
Fin Insp/Yr
1.1
1.6
Wait & counter
12
19
Fines/Yr
Other*
22
24
Workers
Missing
11
0
Chain
12%
10%
$1,602.03
$2,577.11
6.7
5.8
Seats
29.6
29.0
OOB Rate
0.16
0.12
Num of Rest.
29,749
*There are nearly 80 other cuisines offered and 10 other service types not included here
30,796
Mean Taxable Sales by Quarter
350
Mean Entity Sales
(1000s of 2013 $)
300
250
200
150
100
50
0
4
1
2
3
2009
4
1
2
3
2010
4
1
2
3
4
1
2011
2
3
2012
4
1
2
3
2013
Sales Tax Quarter and Year
9
Sample Total Taxable Sales by Quarter
500
Total Sales
(Millions of 2013 $)
450
400
350
300
250
200
150
100
50
0
4
2008
1
2
3
2009
4
1
2
3
2010
4
1
2
3
4
1
2011
Sales Tax Quarter and Year
2
3
2012
4
1
2
2013
3
0
.2
.4
.6
.8
Restaurant Grades By Quarter, Post Grades
0
2
4
6
8
10
Quarter
A
C
B
• Restaurant inspection grades improve after program rollout period
• 80% of restaurants have A grades in the last quarter of 2013
11
Baseline Model, Inspection-Level
•
Y indicates if restaurant i is OOB within a year of an inspection or i‘s fine
liability for an inspection in year t
A = 1 if a restaurant i receives an A for an inspection in year t
C = 1 if a restaurant i receives a C for an inspection in year t
Score is restaurant i‘s inspection score for an inspection in time t
X are restaurant characteristics including:
•
•
•
•
•
•
•
•
•
number of seats, and a series of indicator variables for chain, cuisine, venue type,
and service type
is a zip code fixed effect
is a fiscal quarter-by-year fixed effect
ε is an error term
Standard errors clustered by restaurant
Baseline Model, Group-Level
• Y is the group mean of the log of taxable sales and sales tax liabilities
for restaurants in bin i in quarter t
• A and C are average quarter share with an A or C
•
GP is average quarter share with option to post “grade pending” (for reinspection B or C grades)
• Ungraded is the share of the bin with no grade
• X are mean restaurant characteristics by bin:
•
mean number of seats, and share of bin that is chain, of each cuisine,
venue, and service type, in each borough, zip code, on a corner, and of
each building class (commercial, government, mixed-use, etc)
•
is a bin fixed effect
•
is a fiscal quarter-by-year fixed effect
• ε is an error term
• Some specifications exclude bins with over $1,000,000 in mean
quarterly sales
13
Regression Results, Restaurant OOB
(1)
OOB Within a Year
(2)
(3)
(4)
A
-0.042***
(0.002)
-0.026***
(0.004)
-0.027***
(0.003)
-0.026***
(0.004)
C
0.049***
(0.004)
0.021***
(0.006)
0.016***
(0.006)
0.021***
(0.006)
Constant
0.104***
(0.008)
0.001***
(0.000)
0.076***
(0.009)
0.001***
(0.000)
0.115*
(0.065)
0.001***
(0.000)
0.076***
(0.009)
Q-Y FE
Rest. Char.
Zip FE
Restaurant FE
Inspections
Restaurants
Y
N
N
N
82,977
29,742
Y
N
N
N
82,977
29,742
Y
Y
Y
N
82,977
29,742
Y
N
N
Y
82,977
29,742
Inspection
Score
Low scores are good
Regression Results, Restaurant OOB
(1)
A
C
Constant
Q-Y FE
Rest. Char.
Zip FE
Inspections
Restaurants
Bandwidth
Low scores are good
A-B
(2)
(3)
-0.049*** -0.040*** -0.047***
(0.017)
(0.017)
(0.013)
-0.116***
(0.017)
-0.281*
(0.165)
-0.113*
(0.058)
Y
N
N
Y
Y
Y
Y
N
N
7,387
6,812
1 points
7,387
6,812
1 points
17,113
13,609
2 points
(4)
C-B
(5)
(6)
--
--
--
0.042*
(0.023)
0.039*
(0.024)
0.043***
(0.015)
0.114*** -0.223
(0.006) (0.209)
0.112***
(0.005)
Y
N
N
Y
Y
Y
Y
N
N
2,921
2,921
5,398
2,710
2,710
2,710
1 points 1 points 2 points
Average Fine Per Operating Restaurant
Mean Fines Levied (2013 $)
800
700
600
500
400
300
200
100
0
4
1
2
3
2009
4
1
2
3
2010
4
1
2
3
4
1
2011
Sales Tax Quarter and Year
2
3
2012
4
1
2
2013
3
Regression Results, Inspection Fines
A-B
(1)
A
C
C-B
(2)
-518.46*** -553.00***
(34.53)
(18.19)
__
__
(3)
(4)
__
__
-17.40
(53.29)
100.37**
(39.01)
1129.91*** 1157.09*** 1016.17*** 1127.49***
(46.00)
(29.75)
(124.21) (93.89)
Q-Y FE
Y
Y
Y
Y
Rest. Char.
N
N
N
N
Zip FE
N
N
N
N
Inspections 7,387
17,113
2,921
5,398
Restaurants 6,812
13,609
2,710
4,744
Bandwidth 1 points 2 points 1 points
2 points
Constant
Low scores are good
Mean Sales Tax and Fine Liabilities (2013 $)
Mean Sales Taxes and Fines by Quarter
14000
12000
10000
8000
6000
4000
2000
0
4
1
2
3 4
2009
1
2
3 4
2010
1
2
3 4
1
2
2011
3 4
2012
1
2
3
2013
Sales Tax Quarter and Year
Taxes Levied
Fines Levied
18
Regression results, Sales Taxes
(1)
(2)
(3)
(4)
A
0.0944***
(0.0243)
0.0915***
(0.0249)
C
-0.0930**
(0.0399)
-0.0919**
(0.0410)
-0.0257
(0.0302)
-0.0000
(0.0008)
7.9573***
(0.2369)
Y
Y
Y
-0.0277
(0.0302)
-0.0000
(0.0008)
7.9549***
(0.2425)
Y
Y
Y
Ungraded
Mean Inspection
Score
Constant
Q-Y FE
Bin FE
Restaurant Char.
Exclude High
Revenue Bins
Inspections
Restaurants
-0.0026***
(0.0007)
8.0505***
(0.2367)
Y
Y
Y
-0.0026***
(0.0007)
8.0464***
(0.2422)
Y
Y
Y
N
N
Y
Y
9,647
1,684
9,281
1,627
9,647
1,684
9,281
1,627
Regression results, Taxable Sales
(1)
(2)
(3)
(4)
A
0.0931**
(0.0364)
0.0879**
(0.0374)
C
-0.1000*
(0.0598)
-0.0909
(0.0614)
-0.0239
(0.0453)
-0.0007
(0.0012)
11.0858***
(0.3553)
Y
Y
N
9,281
1,627
-0.0235
(0.0466)
-0.0008
(0.0013)
11.0842***
(0.3633)
Y
Y
Y
9,281
1,627
Ungraded
Mean Inspection
Score
Constant
Q-Y and Bin FE
Restaurant Char.
Exclude High Rev
Inspections
Restaurants
-0.0033***
(0.0010)
11.1775***
(0.3545)
Y
Y
N
9,647
1,684
-0.0032***
(0.0010)
11.1712***
(0.3624)
Y
Y
Y
9,647
1,684
Regression results, With Grade Pending
Sales Taxes
(1)
(2)
Taxable Sales
(3)
(4)
A
0.1348***
(0.0255)
0.1334***
(0.0262)
0.1459***
(0.0383)
0.1420***
(0.0393)
C
-0.1113***
(0.0400)
-0.1111***
(0.0411)
-0.1240**
(0.0600)
-0.1157*
(0.0616)
Grade Pending
0.1545***
(0.0306)
0.1624***
(0.0314)
0.2020***
(0.0459)
0.2092***
(0.0471)
0.0173
(0.0314)
-0.0002
(0.0008
7.9272***
(0.2366)
Y
Y
N
9,647
2,107
0.0169
(0.0323)
-0.0003
(0.0008
7.9230***
(0.2422)
Y
Y
Y
9,281
2,042
0.0324
(0.0471)
-0.0010
(0.0012)
11.0464***
(0.3550)
Y
Y
N
9,647
2,107
0.0338
(0.0483)
-0.0011
(0.0013)
11.0431***
(0.3629)
Y
Y
Y
9,281
2,042
Ungraded
Mean Insp Score
Constant
Q-Y and Bin FE
Restaurant Char.
Exclude High Rev
Inspections
Restaurants
Potential Threats
• Restaurants with “B” inspections might be unobservably different than
those with “A” inspections in ways not reflected in inspection scores
• Restaurants less capable of “getting over the hump”
• Restaurants more capable of winning adjudication
• Some inspected entities are not primarily restaurants
• Estimates unbiased if inspection grades only affect food/beverage sales
• Future analysis will limit sample to primarily restaurant establishments
• Might be other changes to the food environment during this period that
could differentially affect restaurants
• Calorie labeling for chain restaurants
• Changing dietary habits
• Economic changes
Next Steps
• Test sensitivity to different measures of grades
• Inspection grades
• Posted grades
• Grade at the beginning/end of a quarter
• Grade changes from adjudication
• Grade changes from inspection
• Test robustness of sales findings to new sample and new groups
• Restrict entities using food and beverage sales NAICS codes
• Exclude very high revenue entities
• New block random assignment on quarters of operation
• Estimating the differential effect on
• Chains versus individually owned
• Different neighborhoods
• Different cuisine types
Thank You
Michah Rothbart
[email protected]
The results and opinions shown are entirely of the authors, but this
research would not have been possible without the contributions of:
NYC Department of Health and
Mental Hygiene
NYC Department of Finance
Dan Kass
Zachary Papper
Wendy McKelvey
Karen Schlain
J. Bryan Jacobson
Steven Robinson
Melissa Wong
Corinne Schiff
24
INSPECTION
PROCESS WITH
COUNTS
26
Simplified Inspection Cycle Model, Post-Grading
25,969
23,087
11,057
23,660
Initial
Inspection
Re-inspection
88,575
57,008
Adjudication
33,921
17,016
Adjudication
62,659
10,261
5,848
Fines assessed for violations at each inspection that does not lead to an A
LOCAL LINEAR RD
ESTIMATES, OOB
AND FINES
28
0
.2
.4
.6
.8
1
A - B Discontinuity
-20
-10
0
10
Assignment variable relative to cutoff
20
29
.04
.06
.08
.1
.12
Log-linear Regression Results, OOB, A - B
-20
-10
0
10
Assignment variable relative to cutoff
Bandwidth: 1.293, Wald Estimate: -0.0485
20
30
0
500
1000
1500
Log-linear Regression Results, Fines,
A-B
-20
-10
0
10
Assignment variable relative to cutoff
Bandwidth: 3.774, Wald Estimate: -609.21
20
31
0
.2
.4
.6
.8
1
C – B Discontinuity
-10
0
10
Assignment variable relative to cutoff
20
32
.1
.12
.14
.16
.18
Log-linear Regression Results, OOB, C - B
-10
0
10
Assignment variable relative to cutoff
Bandwidth: 1.939, Wald Estimate: 0.0422
20
33
500
1000
1500
2000
Log-linear Regression Results, Fines,
C-B
-10
0
10
Assignment variable relative to cutoff
Bandwidth: 10.35, Wald Estimate: 22.32
20
34
DIFFERENCE-INDIFFERENCE
ESTIMATES, SALES
AND SALES TAXES
35
Regression results, Daily Sales, Dif-in-Dif
A
(1)
Level
244.03
(220.32)
(2)
Level
27.20
(45.07)
(3)
Log
0.1013***
(0.0339)
(4)
Log
0.0966***
(0.0348)
C
-371.47
(359.85)
-25.70
(73.57)
-0.1139**
(0.0553)
-0.1058*
(0.0568)
-478.52
(294.65)
1,898.57
(2,298.84)
Y
Y
N
9,647
1,684
-76.79
(60.41)
1,502.15***
(468.45)
Y
Y
Y
9,281
1,627
-0.0230
(0.0453)
11.0638***
(0.3534)
Y
Y
N
9,647
1,684
-0.0225
(0.0466)
11.0608***
(0.3614)
Y
Y
Y
9,281
1,627
Ungraded
Constant
Q-Y and Bin FE
Restaurant Char.
Exclude High Rev
Inspections
Restaurants
Regression results, Daily Sales, Dif-in-Dif
(1)
Level
(2)
Level
(3)
Log
(4)
Log
A
266.72
(235.27)
81.24*
(48.04)
0.1565***
(0.0361)
0.1533***
(0.0370)
C
-383.34
(362.45)
-54.36
(74.06)
-0.1428**
(0.0557)
-0.1359**
(0.0571)
Grade Pending
82.17
(298.71)
197.32***
(61.01)
0.2001***
(0.0459)
0.2070***
(0.0470)
-455.45
(306.37
1,879.26
(2,300.05)
Y
Y
N
9,647
1,684
-22.36
(62.67)
1,455.18***
(468.38)
Y
Y
Y
9,281
1,627
0.0332
(0.0470)
11.0168***
(0.3532)
Y
Y
N
9,647
1,684
0.0346
(0.0483)
11.0115***
(0.3611)
Y
Y
Y
9,281
1,627
Ungraded
Constant
Q-Y and Bin FE
Restaurant Char.
Exclude High Rev
Inspections
Restaurants