STUDY OF THE SOCIO-ECONOMIC IMPACT OF CERN HL-LHC AND FCC-HH
Workshop on
“The economic impact of CERN colliders:
technological spillovers, from LHC to HL-LHC and beyond”
May 31st, 13:30 – 15:30
Intercontinental Hotel, BERLIN
ARE CERN SUPPLIERS DIFFERENT?
A QUASI-EXPERIMENT
Andrea Bastianin
(University of Milan)
with
Paolo Castelnovo (University of Milan), Massimo Florio (University of Milan) and
Anna Giunta (Center Manlio Rossi-Doria, Roma TRE University)
Outline
• Introduction
• Data
• Methods
• Results
• Conclusions
Introduction
Aim
1.
2.
Identify the determinants of the probability of being a CERN supplier.
Evaluate the impact of CERN procurement on profitability.
Distinguishing features of the study:
1.
2.
Reliance on counterfactual: what would happen to the profits of firms that
are not CERN suppliers, if they were suppliers?
Data about CERN potential suppliers: firms registered with CERN
procurement that have not delivered any order yet.
Relevance:
• A growing body of research investigates the economic impacts of Public
Procumement of Innovation (PPI).
Overview of results
• “Play by the rules”: CERN’s procurement practices
drive the probability of becoming suppliers.
• “Variety matters”: more diversified firms are more likely
to get orders.
• “Innovate”: high-tech firms are more interesting for big-
science labs and hence get orders more easily.
• “Science pays back”: CERN’s effect on profits is
positive and statistically significant.
Data
(with a focus on potential suppliers)
Details about CERN activity codes
http://procurement.web.cern.ch/
Details about CERN activity codes
Details about CERN activity codes
http://procurement.web.cern.ch/
1
…
…
2
CERN ACTIVITY CODES
CIVIL ENGINEERING AND BUILDINGS
ELECTRICAL ENGINEERING
…
…
26
261
262
263
264
265
MAGNETS
Complete large (> 50 tons) magnets
Complete small- and medium-sized magnets
Yokes/Magnets
Coils/Magnets
Superconducting coils/Magnets
…
…
Details about technological intensity of activity codes
• Classification of activity codes according to their technological intensity based
on CERN experts’ assessment of a sample of 300 LHC orders > 10,000 CHF1.
• 23 two-digits hi-tech activity codes
in classes 3-5.
Lo-Tech
(1) very likely to be "off-the-shelf" orders with low
technological intensity
Hi-Tech
(3) mostly "off-the-shelf" but usually high-tech and
requiring some careful specification
(2) "off-the-shelf" orders with an average technological
intensity
(4) high-tech orders with a moderate to high
specification activity intensity to customize products
for LHC
(5) products at the frontiers of technology with an
intensive customization work and co-design involving
CERN staff.
Details in: Florio, M., S. Forte and E. Sirtori (2016). “Forecasting the socio-economic impact of the Large Hadron Collider: A cost–
benefit analysis to 2025 and beyond.” Technological Forecasting and Social Change, 112, 38–53.
1
Potential suppliers
• Who would like to collaborate with CERN?
• Potential suppliers: firms registered with CERN
Procurement Office that never delivered any order.
• In the 1991-2014 period: 2553 firms.
• A total of 7919 self-reported activity codes have been
recorded spanning 59 CERN’s 2 digits codes.
Potential suppliers by country of origin
(% of firms recorded as potential suppliers)
Potential suppliers by self-reported activity codes
Share of Hi-tech and Lo-tech activity codes
Estimation sample
• Actual suppliers (AS): 867 firms that have delivered LHC-
related orders (> 10,000 CHF) between 1991 and 2008.
• Potential suppliers (PS): 2553 firms registered with CERN
Procurement Office that never delivered any order in the 19912014 period.
• Matching CERN data and ORBIS data (with company
accounts data):
• 886 firms (26% of original sample).
• AS = 323 firms (36% of total).
• PS = 563 firms (64% of total).
Estimation sample: timing
ATE estimation
1991
1995
2008
Procurement
Registration
2010
2014
Data: controls
• Controls xt are a set of time-invariant and time-varying variables
belonging to two classes.
1.
Firms’ characteristics:
• Size: Log Total Assets (average over 2008-10);
• Reliability: Liquidity ratio (average over 2008-10);
• R&D intensity: Intangible Assets as % fixed assets (average over 2008-10).
See e.g. Chan et al. 2001, JoF;
• Location: country fixed effects;
• Common idiosyncratic shocks: time fixed effects (for registration year).
Data: controls
2.
CERN’s procurement practices:
• Variety: Dummy = 1 if self-declared no. of activity codes > 1.
• Proximity: Dummy = 1 if firm is located in Switzerland or France.
• Procurement policy: Dummy = 1 for very poorly balanced countries (if Ind.
Ret. Coef. for supplies < 0.3 when firm registered).
• Firms in very poorly balanced Memeber States have a priority in tendering preocedures for orders
> 100,000 CHF.
• Two alternative proxies of technological intensity:
1.
2.
Dummy = 1 if at least 1 of the self-reported act. code is classified as hi-tech.
Categorical variable with share of self-reported hi-tech act. codes (0%, 1%-25%,
26%-50%, …, 100% of total registered activity codes).
Methods
A program evaluation look at CERN procurement
• Aim of program evaluation (PE): to measure the impact of
treatment on outcome for a set of statistical units.
• Units = firms;
• Treatment = being a LHC supplier;
• Outcome = Earnings Before Interest and Taxes, EBIT in 2010 ( profits)
• Characteristics of the methodology:
• There is a vast literature on the econometrics of PE
• James J. Heckman won the Nobel in 2000 for his contributions to the PE.
• The survey by Imbens and Wooldridge (2009, JEL) cites over 300 references.
Propensity score matching in a nutshell
1.
Use a model for binary dependent variables (i.e. logit) to estimate the probability of being supplier:
p(xt) = Pr(treatment | xt).
where xt are control variables capturing firms’ characteristics and CERN’s procurement practices.
2.
Match actual and potential searching “intersections” in their estimated propensity score, p(xt).
3.
Estimate Average Treatment Effect (ATE)
where ATE is difference in EBIT between actual and potential suppliers, once the characteristics that
determine assignment to treatment have been accounted for.
• Aim of steps 1-2 is to approximate a sample design, where firms are randomly assigned to treatment.
Results
Estimates of the probability of being CERN suppliers
CERN
FIRM
Logit model for binary dependend variables
Avg. log Total Assets in 2008-10
Avg. Liquidity ratio 2008-10
Avg. Int. Ass. as % fix. asset 2008-10
1 if No. activity codes>1, 0 if = 1
1 for Switzerland & France
1 if very poorly balanced
1 if Hi-tech (at least 1 act. code)
Share Hi-tech Act. Codes (Categorical)
Observations
Overall
Country Effects
Time Effects
Firm Effects
CERN Effects
(1)
-0.039
-0.035
0.004
0.444*
2.172***
1.140**
0.395*
860
0.0000
0.0000
0.0000
0.5995
0.0000
(2)
-0.041
-0.030
0.005
0.526**
2.251***
1.415***
0.163***
860
0.0000
0.0000
0.0000
0.4634
0.0000
Notes: *** p<0.01, ** p<0.05, * p<0.1. Standard errors clustered by country (i.e. allowing for intra-country correlation). All
models include country and time fixed effects and a constant. The last four lines show p-values for the following tests: ''Overall''
tests the overall significance of the model; ''Country effects'' is a test of the joint significance of country dummy variables; ''Firm
Effects'' is a test of the joint significance of variables in rows 1-3;.''CERN Effects'' is a test of the joint significance of variables
in rows 4-6 and 7 (or 4-6 and 7).
Estimated probability of being CERN suppliers (Model 1)
Lo- VS Hi-Tech firms in well/poorly balanced and very poorly balanced countries
80
100
Hi-tech firms
60
40
0
20
0
20
40
60
Prob(CERN supplier | X), %
80
100
Lo-tech firms
Well or poorly
balanced country
Very poorly
balanced country
Well or poorly
balanced country
Prob(CERN supplier | n. act. codes > 1, non-neighborhood country, at avg. total assets, solvency ratio and IFA-TFA ratio.)
Very poorly
balanced country
Estimated probability of being CERN suppliers (Model 1)
Lo- VS Hi-Tech firms in well/poorly balanced and very poorly balanced countries
DProbability
Base case
80
100
Hi-tech firms
60
40
0
20
0
20
40
60
Prob(CERN supplier | X), %
80
100
Lo-tech firms
Well or poorly
balanced country
Very poorly
balanced country
Well or poorly
balanced country
Prob(CERN supplier | n. act. codes > 1, non-neighborhood country, at avg. total assets, solvency ratio and IFA-TFA ratio.)
Very poorly
balanced country
Estimated incremental probability of being CERN suppliers
With respect to lo-tech firms in a well or poorly balanced country (Model 1)
15
10
0
5
w.r.t. base case, %
Change in Prob(CERN supplier | X)
15
10
0
5
20
Hi-tech firms
20
Lo-tech firms
Well or poorly
balanced country
Very poorly
balanced country
Well or poorly
balanced country
Very poorly
balanced country
Change in Prob(CERN supplier | n. act. codes > 1, non-neighborhood country, at avg. total assets, solvency ratio and IFA-TFA ratio) with respect ot base case.
Base case is a lo-tech firm in a well or poorly balanced country.
Estimates of the CERN effect
With Hi-tech dummy
Estimates of average CERN effect using EBIT 2010 and hitech dummy variable (ATE, Millions of EUR)
Matching method
Suppliers
Potential
ATE
Radius (r = 0.05)
319
151
16.433**
Radius (r = 0.025)
319
146
16.657**
Nearest Neighbor
319
81
13.594*
Kernel
319
151
17.025**
Notes: standard errors based on 500 bootstrap samples. Kernel matching relies on
a Gaussian kernel with bandwidth parameter equal to 0.06.
Conclusions
• LHC procurement is associated with an increase in the profitability of firms.
• Positive impact on EBIT.
• As a side effect propensity score matching delivers some insights on the
CERN procurement strategy.
• The way ahead…
• Quantile approach: Estimate the entire distribution of ATE for EBIT and other
measures.
• Work on the timing of the analysis.
• Add robustness checks: change x and implement different matching algorithms.
27/26
Additional Results
Estimates of the probability of being CERN suppliers
CERN
Firm
Logit model for binary dependend variables – without time fixed effects
Avg. log Total Assets in 2008-10
Avg. Liquidity ratio 2008-10
Avg. Int. Ass. as % fix. asset 2008-10
1 if No. activity codes>1, 0 if = 1
1 for Switzerland & France
1 if very poorly balanced
1 if Hi-tech (at least 1 act. code)
Share Hi-tech Act. Codes (Cat.)
Observations
Overall (p-value)
Country Effects (p-value)
Firm Effects (p-value)
CERN Effects (p-value)
(1)
(2)
0.064**
0.026*
0.005
0.784***
2.638***
2.294***
0.759***
0.060*
0.032**
0.006
0.921***
2.710***
2.323***
855
0.0000
0.0000
0.0118
0.0000
0.231***
855
0.0000
0.0000
0.0096
0.0000
Notes: *** p<0.01, ** p<0.05, * p<0.1. Standard errors clustered by country (i.e. allowing for intra-country correlation). All models include
country fixed effects and a constant. The last four lines show p-values for the following tests: ''Overall'' tests the overall significance of the model;
''Country effects'' is a test of the joint significance of country dummy variables; ''Firm Effects'' is a test of the joint significance of variables in
rows 1-3; ''CERN Effects'' is a test of the joint significance of variables in rows 4-6 and 7 (or 4-6 and 7).
Estimates of the «CERN effect»
With Hi-tech categorical variable – with time fixed effects
Estimates of average CERN effect using EBIT 2010 and hi-tech
categorical variable (ATE, Millions of EUR)
Matching method Suppliers Potential N
ATE
t-stat
Radius (r = 0.05)
319
154
473 16.473** (2.3758)
Radius (r = 0.025) 319
149
468 16.525** (2.1150)
Nearest Neighbor 319
83
402 17.878** (2.4464)
Kernel
319
154
473 16.990** (2.5747)
Notes: standard errors based on 100 bootstrap samples. Kernel matching
relies on a Gaussian kernel with bandwidth parameter equal to 0.06.
CERN
FIRM
Propensity score estimation: controls, xt
concept
variable
Size
Avg. log Total Assets in 2008-10
Reliability
Avg. Liquidity ratio 2008-10
R&D intensity
Avg. Int. Ass. as % fix. asset 2008-10
Location
country fixed effects
Common shocks
time fixed effects (for registration year)
Variety
1 if No. activity codes>1, 0 if = 1
Proximity
1 for Switzerland & France
Procurement policy
1 if very poorly balanced
Tech. Intensity
1 if Hi-tech (at least 1 act. code)
Tech. Intensity
Share Hi-tech Act. Codes (Categorical)