Composite Indicators for
the Measurement of
Economic Performance
P. Roberti
F. Oropallo
ISTAT
Productivity, Competitiveness and the New Information Economy
Business , Systemic and Measurement Issues
NESIS FP5
ISTAT – Rome
June 26, 2003
We live with too many
Indicators
LISBON OBJECTIVES
 “to become the most
competitive and dynamic
knowledge-based economy
in the world, capable of
sustainable economic
growth ...and social
cohesion”
June, 26 2003
A host of indicators that have
been proposed to monitor
this goal.
They can be grouped in five
areas:

General economic background
Employment
Economic Reform
Social Cohesion

Environment



Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
2
Indicators can be
one-dimensional or
multi-dimensional
Composite indicators have received increasing
attention in recent years.
Various methodologies have been developed to
handle aggregation and related problems:






Aggregation systems
Deciding on the phenomenon to be measured
Selection of sub-indicators
Assessing the quality of the data
Assessing the relationships between the sub-indicators
Testing for Robustness and Sensitivity
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
3
State of the Art –
Methodological Issues
A number of methodologies can be applied for the
development of composite indicators. They include:









Multiple linear regression models
Principal components analysis and factor analysis
Cronbach alpha
Neutralization of correlation effect
Efficiency frontier
Distance to targets
Experts opinion (budget allocation)
Public opinion
Analytic Hierarchy Process
JRC – EC (2002) - State-of-the-art Report on Current Methodologies and Practices for Composite Indicator Development –
Joint research Centre – European Commission - Institute for the Protection and Security of the Citizen Technological and
Economic Risk Management I-21020 Ispra (VA) Italy - Prepared by the Applied Statistics Group June 2002
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
4
Is there a clear Framework yet?
How can a framework be developed?
By defining an analytical framework with
precise properties!
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
5
Property of Indicators
•
•
•
•
•
•
•
June, 26 2003
Micro founded
Scope fulfilling
Purpose oriented
Well- behaved
Consistent
Decomposable
Multidimensional - Composite
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
6
MICRO FOUNDED
that is, based on a comprehensive database that embraces all
aspects of enterprise features (ad hoc surveys can cover some
aspects)
 Integration of different data sources of micro data
 Quality test of the integration process (matching procedures / estimation)

In the second part of this presentation examples of the
opportunities opened up by an Integrated Database are shown
(Diecofis Project www.istat.it/diecofis Year of reference is 1998-2000)
Sources are: (1) Structural Business Statistics (2) Administrative data (Foreign Trade,
Commercial account, Fiscal and Social security data)
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
7
SCOPE FULFILLING






Indicators can measure size, change and dispersion
Changes in an Indicator’s value can have different causes and
lead to different conclusions depending on underlying
combinations
Socio-economic phenomena have different and complex
dimensions.
Different indicators can serve different purposes, i.e. measure:
- heterogeneity/dispersion
-performance (moving toward the mean)
- overall systemic performance
ß-performance (generalised move upward/downward)
overall comparative performance (e.g. catching up/lagging
behind)
stratification (to evaluate differences in systemic structures and
whether they represent a “stratum” - as with Yitzhaki’s
decomposition)
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
8
PURPOSE ORIENTED


Socio-economic phenomena may have many facets and
change can result from a combination of different patterns
Appropriate indicators may be needed in different
circumstances
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
9
WELL-BEHAVED AND CONSISTENT


Indicators inconsistency may arise for different
reasons
Reference to condition of
- Lorenz dominance (focusing on relative
differences)
- Pareto superiority (focusing on levels)
- stochastic dominance (focusing on both
dimensions)
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
10
DECOMPOSABLE &
MULTIDIMENSIONAL

To be able to
 study patterns
 take into consideration more than one
dimension/aspects
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
11
Developments in poverty analysis
are a good example of the possible
problems and avenues to solve
them
Number of people  Income gaps  Welfare dimensions  Multidimensional aspects
Headcounts  Gaps  FGT indices (squared gaps, etc.) 
June, 26 2003
Multidimensional indices
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
12
Developing a similar framework is
important for the analysis of
systemic performance and the
benchmarking of “economic
textures”
Since many factors and forces are at work to
determine, condition and produce different outcomes.
The quote that follows can serve to grasp the problem
and possible approach to address it
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
13
“The fact that many of the smaller EU economies do either better or
worse than the larger ones is partly due to larger EU economies
contributing more to the overall EU mean than smaller economies, which
means that they are less able to diverge from the mean. A second
explanation is due to structural conditions. The industrial distribution of
small economies is often concentrated in a few sectors, while larger
economies are more diverse. This can shift the scores towards the mean
for many indicators in large economies, while small economies can
exhibit either a high or low innovative capacity, depending on the sectors
that dominate the economy. Of course, this shift towards high or low
technology sectors is not accidental, but reflects both public and private
institutions seeking out areas of comparative advantage and high
profitability”.
(EU Commission)
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
14
The “analytical framework” and
indicators that can serve to study
and benchmark different
levels/dimensions of “systemic
features”








Systemic maps (whole jigsaw)
The overall aggregate picture: EU level
The national, regional, local picture
The sectional picture
The occupational picture
Systemic strength and weakness, at a point in time (cross-section
analysis) and overtime (longitudinal analysis)
Map transitions: features, patterns and evolution (“New” vs.
“Old”)
Systemic change and its features, at the aggregate/disaggregate
levels
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
15
Decomposable indicator how
The GINI index measures the concentration of a particular phenomenon
(0=no concentration, 100=maximum concentration). It can be divided
into three elements:
GINI  within  between  overlap
K
GINI   Gk p k  k 
k 1
June, 26 2003
1

K
K
 ( y
k
k i
k
 y i ) p k pi  L
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
16
Distributions and Overlap
When overlap in the decomposition is high, it is very
hard to judge which group is the best/worst, because
distributions cross
N
Mean (A) ≈ Mean (B) ≈ Mean (C)
June, 26 2003
performance
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
17
Distributions and Overlap
When overlap in the decomposition is low it is easy
to determine which group is the best/worst performer
N
Mean (A)
June, 26 2003
Mean (B)
Mean (C)
performance
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
18
Mono-dimensional Analysis of
performance (Overall)
(GINI index calculated on exports)
Gini Index
TABLE 8 Gini Index-decomposition by sector of activity and localisation
a) Enterprises classified according to sector of activity
1999
1994
Variation
0.92
0.91
0.01
of which:
"within" component
"between" component
Overlap
Gini Index
0.064
0.477
0.383
0.067
0.419
0.428
0.00
0.06
-0.04
b) Enterprises classified according to regional localisation
1999
1994
Variation
0.92
0.91
0.01
of which:
"within" component
"between" component
June, 26 2003
Overlap
0.136
0.143
0.159
0.144
Composite
Indicators for the Measurement
of
Economic Performance - Roberti / Oropallo
0.628
0.626
-0.01
0.02
0.00
19
Mono-dimensional analysis of
performance (Exports)
S ectors*
DK
DM
DG
DB
DJ
DL32
DA
DH
DL
DC
DN
DF
DI
DL30
DE
DL313
I
DL332
K
G
Gini Index (a)
0.88
0.95
0.91
0.88
0.90
0.95
0.90
0.89
0.89
0.86
0.88
0.94
0.91
0.97
0.94
0.80
0.96
0.89
0.95
0.90
(By NACE Sectors)
Contribution of within
component %(b)
1.2
0.1
0.1
0.6
0.6
0.0
0.2
0.1
0.1
0.2
0.3
0.0
.01
0.0
0.1
0.0
0.0
0.0
0.0
2.6
Contribution of Between component (c)
1999
1994
10.8
9.0
9.0
6.7
6.3
4.9
3.5
3.8
2.8
2.3
2.0
1.6
1.9
1.8
1.8
1.3
1.7
1.2
1.6
1.9
1.5
1.3
1.2
1.0
1.0
1.0
1.0
1.7
0.5
0.4
0.2
0.2
0.2
0.6
0.2
0.1
0.2
0.3
0.1
0.1
a) intra-group Gini Index
b) Specific Gini Index adjusted for the weight of every sector
c) adjusted index of the difference between average exports of each sector. Every sector compares
with sectors with lower average exports.
* for sector acronyms see grapf 13
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
20
Multi-dimensional
analysis of performance
Decomposition for each dimension
G d  w d  b d  Ld
Decomposition of the between component for
each dimension across K classes
K
b  b
d
k 1
June, 26 2003
d
k
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
21
To make composite indicators easy
to use and interpret
Requires Normalization
bn 
d
k
d
k
b
d
k
Max(b )
d  1,..m
Requires Aggregation
1 m
C   wd bnkd
m d 1
with 0  wd  1 and
w
d
1
d
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
22
Benchmarking performance levels
or gaps?
How far away from best performers enterprises are?
Gapkd  100  bn kd
Gapkd  100 
bkd
Max (bkd )
That is :
d
d
b

)
b
(
Max
d
k
k
Gapk  100
Max (bkd )
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
23
Weighted Composite Gap
How far away from best performers enterprises are?
1 m
Cgap   wd Gapkd
m d 1
with 0  wd  1 and
w
d
1
d
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
24
Composite Indicator
(Breakdown by NACE sector)
Three dimensions
Sector
I
E
DK
DF
DJ
DG
DL
DM
DA
DB
DE
K
DI
DH
DN
DC
C
G
F
DD
H
Value Added
100
62
55
53
49
41
34
33
31
27
22
22
20
20
10
9
9
6
3
1
-
June, 26 2003
Sector
I
DJ
DK
DB
DL
DM
DA
DE
DG
H
DH
DI
E
DN
F
DC
DD
DF
K
C
G
Employment
100
74
70
63
47
43
39
35
32
28
26
25
24
23
22
21
6
4
4
4
2
 One Dimension
Sector
DK
DM
DG
DL
DB
DJ
I
DC
DN
DA
G
DH
DI
DF
DE
K
DD
F
E
C
H
Export
100
69
49
46
43
43
33
33
26
23
22
20
16
9
7
2
2
1
0
0
0
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
Sector
I
DK
DJ
DM
DB
DL
DG
DA
E
DH
DF
DE
DC
DI
DN
G
H
K
F
C
DD
Composite
78
75
55
48
44
43
41
31
29
22
22
21
21
21
20
10
9
9
9
4
3
25
NACE sectors
ACTIVITY DESCRIPTION
C
PRODUCTS FROM MINING AND QUARRYING
DA
FOOD PRODUCTS, BEVERAGES AND TOBACCO
DB
DC
DD
TEXTILES AND CLOTHING INDUSTRY PRODUCTS
LEATHER AND LEATHER PRODUCTS
WOOD AND PRODUCTS OF WOOD AND CORK (EXCEPT FURNITURE)
DE
DF
DG
PULP, PAPER AND PAPER PRODUCTS; RECORDED MEDIA; PRINTING SERVICES
COKE, REFINED PETROLEUM PRODUCTS AND NUCLEAR FUEL
CHEMICALS, CHEMICAL PRODUCTS AND MAN-MADE FIBRES
DH
DI
RUBBER AND PLASTIC PRODUCTS
OTHER NON METALLIC MINERAL PRODUCTS
DJ
DK
DL
BASIC METALS AND FABRICATED METAL PRODUCTS
MACHINERY AND EQUIPMENT N.E.C.
ELECTRICAL AND OPTICAL EQUIPMENT
MACHINERY AND COMPUTERS
Insulated wire and cable
RADIO, TELEVISION AND COMMUNICATION EQUIPMENT AND APPARATUS
Instruments and appliances for measuring, checking, testing, navigating
Industrial process control equipment
TRANSPORT EQUIPMENT
OTHER MANUFACTURED GOODS N.E.C.
DL30
DL313
DL32
DL332
DL333
DM
DN
E
F
G
G5143
G5164
G5165
H
I
I642
J
K
K7133
K72
M - N -O
ELECTRICAL ENERGY, GAS, STEAM AND WATER
CONSTRUCTION WORK
WHOLESALE AND RETAIL TRADE SERVICES
Wholesale trade services of electrical household appliances and radio and television goods
Wholesale trade services of machinery and computer
Wholesale trade services of Industrial equipment
HOTEL AND RESTAURANT SERVICES
TRANSPORT, STORAGE AND COMMUNICATION SERVICES
Telecommunications services
FINANCIAL INTERMEDIATION SERVICES
REAL ESTATE, RENTING AND BUSINESS SERVICES
Renting services of office machinery and equipment including computers
COMPUTER AND RELATED SERVICES
EDUCATION – HEALTH AND SOCIAL - OTHER SOCIAL SERVICES
The Sectors in italic are defined “ICT sectors” by OECD
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
26
Composite Indicator
(Breakdown by NACE sector)
Three dimensions of enterprises’ performance: (1)Value
Added (2)Employment (3)Exports
100
a) one dimension-three areas
90
VALUE ADDED
80
EMPLOYMENT
EXPORT
70
60
50
40
30
20
10
DK
I
DM
June, 26 2003
DJ
DG
DL
DB
DA
DC
DN
DH
E
DF
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
DI
DE
G
K
F
H
C
27
DD
Composite Indicator
(Breakdown by NACE sector)
COMPOSITE
100
b) 3 dimensions into one indicator
90
80
70
60
50
40
30
20
10
0
I
DK
June, 26 2003
DJ
DM
DB
DL
DG
DA
E
DH
DF
DE
DC
DI
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
DN
G
H
K
F
C
DD
28
Composite Indicator
(Breakdown by regions)
Three dimensions of enterprises’ performance: (1)Value
Added (2)Employment (3)Exports
100
a) 3 mono-dimensional indicators
VALUE ADDED
80
EMPLOYMENT
EXPORT
60
40
20
LOM
LAZ
June, 26 2003
PIE
VEN EMR TOS
FVG
TAA MAR
LIG
ABR
SAR PUG UMB CAM VDA
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
BAS
SIC
MOL
CAL
29
Composite Indicator
(Breakdown by regions)
100
Composite Indicator
b) from 3 to one 3-dimensional indicator
80
60
40
20
LOM
LAZ
June, 26 2003
PIE
VEN
EMR
TOS
FVG
TAA
MAR
LIG
ABR
SAR
PUG
UMB
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
CAM
VDA
BAS
SIC
MOL
CAL
30
Composite Indicator
(by size of firm)
Three dimensions of enterprises’ performance: (1)Value
Added (2)Employment (3)Exports
COMPOSITE
100
VALUE ADDED
80
100
EMPLOYMENT
80
EXPORT
60
60
40
40
20
20
-
-
>99
June, 26 2003
10-49
50-99
1-2
3-9
>99
10-49
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
50-99
1-2
3-9
31
Conclusions
The analysis which has been presented draws from research results under two
related FP5 projects DIECOFIS and NESIS that deal with different mapping and
benchmarking aspects.
A strong investment in the design and development of a complex and wide ranging
system of enterprise micro data which have been integrated and systematised
into one single “hub”.
The analysis is founded on micro-data drawn from the integrated and systematised
enterprise SIS, which gives high flexibility and allows to aggregate and
disaggregate indicators a la carte.
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
32
Thank You
June, 26 2003
Composite Indicators for the Measurement of
Economic Performance - Roberti / Oropallo
33