lessons for evaluation practice and for policy design from the

LESSONS FROM THE EVALUATION
OF TWO ITALIAN ENTERPRISE
SUPPORT PROGRAMMES
Daniele Bondonio
Alberto Martini
et al.
The two programmes
One national, targeted to all firms but
focused on the South
One regional (the combination of 29
measures in Piemonte, for which
only SMEs were eligible)
PIEMONTE
Torino
Fiat
Holy Shroud
2006 WinterOlympics
Home of the 2010 CIE Summer School at PRACATINAT
Home of the 2012 CIE Summer School (Aug 30-Sep 9)
Law 488/92
A nation-wide programme targeted to industrial firms in
disadvantaged areas, to support investments in physical
capital, providing non-repayable grants
No limit on firm size, but mostly small firms
50% of grant recipients < 10 employees, only 1% > 500
Average grant size € 420,000
Total outlays € 2.6 billion in 2000-2006
29 ways to help SMEs in PIEMONTE
multitude of support measures for SMEs in Piemonte
About 10,000 recipients
1500 capital grants (14%)
7800 interest-rate grants (75%)
1000 soft-loans (10%)
Average equivalent value of subsidies = € 10,600
Total resources spent in 2005-09 = € 120 million
What are the effects of these programmmes
on employment, sales and investment?
Which difference does public support make
for different firms?
Which evidence can be trusted for future
decisions?
Used a rigorous approach
Jobs due to the subsidy
=
change in employment among subsidized firms
−
change in employment among subsidized firms
if they had not been subsidized
(the latter is known as counterfactual, and can never be observed)
IN PRACTICE WHAT IS DONE IS TO FIND A PROXY FOR THE
COUNTERFACTUAL
Jobs due to the subsidy
=
change in employment among subsidized firms
change in employment among NON subsidized firms
THAT ARE “IDENTICAL” TO THE SUBSIDIZED FIRMS
IDENTICAL ……. VERY SIMILAR …….. WITH THE SAME OBSERVABLE CHARACTERSITICS
Law 488 used rankings based on five
indicators, one of which was
“jobs to be created”
Rejected and accepted firms share the same willingness to
invest, and the residual differences can be more easily
eliminated through matching techniques
So we used matching techniques on applicants firms
using outcome data provided by ISTAT
AND WE GOT SOME INTERESTING RESULTS
Let’s do the official calculations
Funds disbursed for Law 488
€ 2,600,000,000
“target” jobs (TARGET “ACHIEVED”) 82,000 jobs
€ 31,700
OUR CALCULATIONS
Impact of subsidy = 1.82 additional jobs per firm
Number of firms subsidized =
6,189
Number of jobs really created =
11,236
Cost per job
Cost per job really due to the subsidy = € 231,207
The results are very different for the 29 programs for SME
1.82
Law 488
232, 000
Is this enough evidence to prove that
SMALL IS BEAUTIFUL?
CERTAINLY NOT YET!
Does the size of 488 grants matter?
No. firms used
in the analysis
Average impact
on employment
<125,000
125,000
250,000
250,000 500,000
>500,000
1,702
1,637
1,534
1,222
0.91
1.61
2.38
2.69
577
million
1. 61
billion
158,048
488,676
Total support
received
122
million
Cost per job
created
79,460
296
million
112,252
Is it the size of the firm to make a difference?
Average impact on
employment
Firms used in the
analysis
Total amount of support
received
Cost per job created
ABOVE
250
50-250
10-50
LESS THAN
10
-2.34
2.80
1.89
1.61
51
670
2419
3049
58 mill
442 mill
968 mill
1.18 billion
N.A.
235,590
211,098
230,700
The ways to help SMEs in
PIEMONTE
capital grants
interest-rate grants
soft-loans
WHICH ONE IS MORE COST- EFFECTIVE?
AND THE WINNERS ARE
POLICY IMPLICATION: MAKE NO PRESENTS, HELP THEM
TO GET CHEAPER FINANCING (EITHER WITH LOANS
EITHER PAYING THEIR INTEREST COST)
SUMMARY OF THE RESULTS
Holding everything else constant…..
• Large grants are less cost-effective than smaller grants
• Grants to large firms are not cost effective
• Re-payable assistance is more effective than capital grants
Results suggest that credit market imperfections for SMEs are
the main issue that may justify public intervention
No evidence of effect on the quality of the
jobs generated by the subsidies
Sales and investment results follow the
same pattern of employment
(albeit with some additional volatility
due to data quality)
SHOWS THESE RESULTS TO THE POLICY MAKER
HO HO HO HOLY SHIT!
page
FOR CHRISTMAS WE WILL GET NO PRESENTS,
ONLY SOFT LOANS,
AND THOSE WHO HAVE BEEN REALLY NAUGHTY
WILL GET ITALIAN GOVERNMENT BONDS
HO HO
HO EUR
HO