Urban Public Transport
Evaluation with Conflicting Objectives
T. Suguiy*1,3, M.F.H. Carvalho2, P.A.V. Ferreira3
1
CTI Renato Archer, Campinas, Brazil
2
PUC-Campinas, Brazil
3
FEEC Unicamp, Campinas, Brazil
*Corresponding Author: Takao Suguiy
Rodovia Dom Pedro I (SP-65), Km 143,6 - Campinas, SP
13069-901
Brazil
email: [email protected]
Urban Public Transport
Evaluation with Conflicting Objectives
An efficient urban public transport contributes to improving the quality of life in
the city. With that in mind, public authorities must establish transportation
policies and regulations according to the social interest of this important activity.
An essential aspect for that is the definition of indices to guide the proposition
and supervision of policies established by the authorities. The scarcity of
resources forces the public sector to strive to balance the benefits and costs of the
urban public transport. In particular, it is necessary to compromise the interests of
service providers and users in order to find an adequate equilibrium. The ultimate
goal should be the satisfaction of three key players: service providers, users and
public authorities. The service provider aims to minimize production costs and
maximize return on investments; the user demands for less fare and waiting time,
and more comfort, reliability and flexibility of access to all the points of interest;
it is the duty of public authorities to mediate the conflicting interests of service
providers and users. The urban public transport problem involves conflicting
objectives with multiple inputs and multiple outputs, and due to its inherent
complexity, non-parametric methods seem to represent the best performance
evaluation tools. This paper proposes a two-step methodology to evaluate the
performance of a number of urban public transport systems. The first step
identifies groups of homogeneous systems, that is, cities, or more generally,
units, using Cluster Analysis. In the second step, the efficiency of each unit with
respect to the others in a same group is assessed using Data Envelopment
Analysis (DEA). The methodology has been employed to evaluate the
performance of Brazilian urban transport systems with more than 40,000
passengers transported per day. The methodology is based on measures of
efficiency (the viewpoint of service providers) and effectiveness (the viewpoint
of users). Our results show that urban transportation policies should and can be
established with basis in the simultaneous consideration of efficiency and
effectiveness indices.
Keywords: DEA, cluster, urban public transport, conflicting objectives
1. Introduction
Urban public transport is a service, and as such, it is perishable, intangible, heterogeneous and
takes place only with the presence of the user. The demand for public transport is subject to
seasonal variations and fluctuations in consumption points in the day, week and year. Another
feature of this activity in Brazil is that it can be operated by public or private companies. For the
most part, it is performed by private companies that must achieve an appropriate level of service
quality, regulated and evaluated by public authorities according to the social interest of the
activity. The management of the public transport service considers the interaction between the
user, the company that provides the service and the public authority, which acts as a mediator in
the interest of both users and companies.
The evaluation of public transport management systems can be made according to different
indices, such as flow of vehicles, accessibility, safety and comfort, with the scientific literature
offering different performance indicators. However, the evaluation is subject to detailed
information, as available infrastructure, regulations or legislations, population density,
topography, etc. There is an inherent difficulty in establishing the actions to be developed for
improving the performance of urban public transport, even if the interest resides in macro
indicators that describe the behavior of the whole network system as a function of their
characteristics (Fancello, Uccheddu, & Fadda, 2014). Measuring instruments have greater
significance for assessing performance, not in absolute terms, but in relative terms, when they
allow to compare an urban unit to a set of equivalent units in terms of performance.
Another aspect of interest is the positioning of the three main participants in the system; the
service provider, the public authority and the user. While the service provider aims to minimize
production costs and maximizing return on investment, the user demands, more often, less
waiting time, more comfort and flexibility of access to all points of interest. The objectives of
service provider and the users are usually conflicting. As the urban public transport is a service
provided to the population under concession of the public sector, it is not strictly subject to a
competitive environment (as, for example, the self-regulated retail sector), the public authority
should act as a mediator by establishing parameters and measures that fulfill the basic needs of
the population in terms of mobility, even at the expenses of a non-profit operation (as the use
of a vehicle in a determined schedule, route or time where the number of passengers is very
low). On the other hand, the public authority must also ensure the sustainability of the service
provider.
Considering the three key stakeholders - user, public authority and service provider -, their
interaction can be described as shown in Figure 1: the public authority offers the urban
infrastructure, establishes policies for the sector, and defines and specifies routes and scales for
the companies that provide the service, under supervision. In turn, users receive and pay for the
service, being conferred to them the power of contributing to the public welfare in the
specification and evaluation of the quality of the services provided.
Given the above characteristics and requirements, the urban public transport management must
ensure the efficiency and sustainability of the service provider and, at the same time, the
effectiveness of the system, a measure of how much the services specifications imposed by the
authority have been achieved. In seeking a balance between efficiency and effectiveness,
several variables must be considered: passengers transported, fleet available, distance traveled,
population, local GDP, population density, etc. (Pina, & Torres, 2001). The urban public
transport by bus is a complex system, being very difficult to set goals and develop a
comparative analysis among municipalities (Fancello, Uccheddu, & Fadda, 2014), in this paper
also referred as Decision Making Units (DMUs).
Public
Authority
Service
Provider
User
Figure 1. Relationship in the provision of public transport services
Adapted from Devarajan & Shah, (2004).
The comparative analysis of complex systems can be carried out, basically, by two classes of
methods. Parametric methods assume the existence of a function that represents the relationship
between the variables. As the number of variables can be large and the variables very different
in nature, these methods have limited application in practice (Borger, Kerstens, & Costa, 2002).
On the other hand, nonparametric methods have been widely employed in the evaluation of
urban public transport. Karlaftis, Gleason, & Barnum, (2013) presents a list of ninety-seven
publications related to the application of a non-parametric technique, Data Envelopment
Analysis (DEA), to urban transport problems.
Several studies in different fields use DEA for benchmarking (Morini, Barassa, Maurice,
Moretti, & Porto, 2014), (Soliman, Siluk, Neuenfeldt JR, Married, & Paris, 2014). One of the
conditions for proper application of DEA is that the DMUs constitute an homogeneous set. In
the case of public transport, each city (municipality) has its own characteristics: size,
population, population density, GDP, etc. In search for homogeneity, it is interesting to group
municipalities into clusters with similar characteristics. In this article we discuss the use of
parametric techniques such as clustering analysis to group DMUs (municipalities) together in
homogeneous sets.
An important objective of the urban public transport is to guarantee both the sustainability of
the service provider that operates under concession and the fulfillment of the users needs. A
balance between such usually conflicting goals is pursued by public authorities in the form of
services regulations.
This article proposes a performance evaluation of urban public transportation systems by the
association of two benchmarking indices: the Technical Efficiency Index, which represents the
vision of the service provider, and the Effectiveness Index which represents the interests of the
user. In Section 2, we discuss two different approaches for obtaining homogeneous sets of
municipalities. Then, in Section 3, the concepts of effectiveness and efficiency as key elements
for the definition of performance in urban public transport are introduced. Section 4 presents
previous studies in the area. In Section 5 we apply the methodology to fifteen Brazilian cities
and fifteen world cities. The analysis of the effectiveness/efficiency of DMUs (cities) is held in
two phases. In the first phase, we use parametric techniques for grouping DMUs in
homogeneous sets. In a second phase, DEA provides a performance evaluation of each unit with
respect to peers. The paper includes rankings of the cities when particular performance indices
are considered.
2. Homogeneous sets formation
The formation of homogeneous sets can be accomplished by a variety of techniques and
algorithms. Here, the objective is to find and separate individuals (units) into groups according
to their similarities. Hierarchical methods require the calculation of a matrix of similarity /
distance, given an initial distribution of individuals in a predefined number of groups (Bussab,
Miazaki & Andrade, 1990). The cluster analysis developed in this paper is based on a nonhierarchical method called K-means (MacQueen, 1967), which consists in the transfer of an
individual to the group whose centroid corresponds to the shortest distance. This method is
implemented in most statistical softwares due to its ease of application even for a large number
of individuals.
Table 1 - Data for twenty decision units.
U
1
2
3
4
5
6
7
8
9
10
11 12 13
E1
1
2
3
4
2
3
3
4
5
2.5 6
7
6.2 7
8
9
10 11
E2
5
3
2
1
5
4
8
7
9
10
5
4
4
2
3
5
14 15 16 17 18
3
19
20
10
11
1.2 1.5 2
Table 1 presents the data of twenty decision units with characteristics specified by inputs E1
and E2 (Po, Guh & Yang, 2009). Figure 2a shows the arrangement of these units in the plane
E1xE2. Figure 2b presents the efficient frontier determined by application of DEA when the
objective is to minimize both inputs. Figure 2c shows four possible groups considering the
proximity (similarity) of the characteristics of each unit. Figure 2d proposes a grouping defined
by the dominant units of the efficient frontier. This grouping is delimited by straight lines
connecting the origin and two efficient bondary units. Group A1 is in the area bounded by the
cone formed by the origin and the points U1 and U2. It contains U1, U2 and other 5 units,
which compound to ratio E2/E1 > 1. Group A3 is in the area bounded by the cone formed by the
origin and the points U3 and U4. It contains U3, U4 and other 3 units. The units belonging to
this group are such that E2/E1<1.
Considering the entire data set (twenty DMUs), units U1, U2, U3 and U4 become the
benchmark for all other units (Figure 2d).
12
12
E2
E2
10
10
8
8
6
6
4
4
2
2
Efficient frontier
U1
U2
U3
E1
0
4
2
0
8
6
10
12
E1
U4
0
.a ) Units representa tion in the E1-E2 pla ne
4
2
0
8
6
12
10
.b) Efficient frontier by DEA
12
12
E2
E2
10
A1
10
8
A1
8
A2
A3
6
6
A4
4
A3
4
A2
2
U1
0
0
2
4
6
.c) Proximity by grouping of K-mea ns
8
10
12
A4
U2
2
U3
E1
0
0
2
E1
U4
4
6
8
10
12
.d) Grouping by domina nce of DMU
Figure 2 - Different forms of groupings of twenty units of DMU
The inefficient units in clusters A1, A3 and A4 of Figure 2c are far from the efficient frontier of
Figure 2b. In Figure 3, the efficient frontier of each group, as determined by DEA, are
presented. Four efficient frontiers are defined, one for each group, and all the DMUs on the
frontier of the group are considered efficient with respect to the group.
Whenever the improvement of a unit occurs incrementally, it is more sensible that the analysis
be done with respect to the efficient units of its group.
As an example, unit U10 should seek efficiency, with reference to the efficient set F1, Figure 3.
Each inefficient unit should move towards a local efficient unit.
The efficient frontiers of groups F1, F2, F3 and F4 are only locally efficient. A second step is
required to consider all the DMUs in the efficient frontiers of groups F1, F2, F3 and F4, and
then measure the distance between the clusters, which can be done by applying DEA to all the
units that constitute the efficient frontiers.
3. Efficiency, effectiveness and performance in urban public transportation
systems
Efficiency and effectiveness are central concepts for assessing and measuring the performance
of organizations (Mouzas, 2006). Informally, efficiency is associated with the "the correct way
to do things", while effectiveness with "do the correct things" (Drucker,1997). Efficiency refers
to the ability of an organization to produce the desired output with the minimum input level, or
the maximum output for a given input. Efficiency is related to the best use of resources for
accomplishing the objectives of the organization.
12
E2
10
U10
F1
8
6
U1
F3
U5
U12
U6
4
U2
U3
2
F4
F2
U20
U4
E1
0
0
2
4
6
8
10
12
Figure 3 - Efficient frontiers of the groups.
Effectiveness is a measure of the organization capability to achieve predetermined objectives. It
indicates success or failure in meeting the objectives set by the organization. Performance is
linked to efficiency and effectiveness, and their relationship can be described as in Figure 4 and
5.
Performance
Efficiency
Effectiveness
Figure 4 - Efficiency, effectiveness and performance relationship
A study involving efficiency and effectiveness of two hundred and fifty-six US urban transit
systems over five years (1990-1994) pointed out a positive correlation between efficiency and
effectiveness (Karlaftis, 2004). This study adopted as criteria of efficiency and effectiveness, the
number of vehicles per mile and the number of passengers per mile, respectively. The first
criterion represents the density of the service area; the second criterion, the number of
passengers transported, to be maximized. Both criteria reflect the interest of the service
provider.
Other empirical studies (Ho & Zhu, 2004), (Raphael, 2013) involving banking systems, with
efficiency related to how a bank used its assets, and effectiveness with how the bank obtained
net return, did not find a correlation. Depending on their definitions, efficiency and
effectiveness can be interdependent (Kumar & Gulat, 2010). For example, if the objective is to
minimize cost, efficiency contributes to the objective and is directly related to effectiveness, if
the later is defined by the attainment of the lowest cost product. When the priority of the
consumer is volume, flexibility and variety of products, efficiency has an inverse relationship
with effectiveness.
Efficiency
Efficiency in urban public transport represents the interest of the service provider while
effectiveness represents the extent in which the service provider achieves the objectives
established by the public authority in the benefit of the service user. Thus, an increase in
efficiency may correspond to a decrease in effectiveness, and vice-versa. In Figure 5, focusing
only in effectiveness neglecting the efficiency may result in a non-sustainable return over time.
On the other hand, focusing only in efficiency may result in a service with inadequate quality to
the user, making it also unsustainable over the time.
Effectiveness
Figure 5 - Relationship between effectiveness and efficiency
The public authority, responsible for granting the right of exploitation of public transport,
assumes the position of the regulator, encouraging the maximization of both efficiency and
effectiveness, while maintaining the balance of the system. In other words, the public authority
must induce the companies to operate at the highest possible point on the sustainable return
line.
4. Data envelopment analysis - DEA
DEA is a linear programming approach used to measure relative efficiency in converting inputs
(resources) in outputs (products) of a set of homogeneous units (DMUs).
Consider a set with n decision units, indexed by j = 1, 2, ..., n each using xij inputs indexed by
i = 1, 2, ...,m and generating outputs yrj indexed by r = 1, 2, ...,s. Let ur and vi multipliers
associated with outputs and inputs respectively. The efficiency of the jth decision unit can be
written as:
E 0  max
s
u
r 1
subject to:
m
v
i
i 1
xio

1
r
yr 0
0
=1,2, …, n
(1)
s

r 1
m
u r y rj 
v x
i ij
0
j  1, 2,..., n
i 1
{vi ur }  
r  1,2,...s; i  1,..., m
where yrj and xij (all positive) are the known outputs and inputs of the jth DMU and ur, vi > 0 are
variable weights to be determined by solving the linear problem (1) (Hadad, Keren & Hanani,
2013); ε > 0 is a non-Archimedean infinitesimal number (Charnes, Cooper & Rhodes, 1978).
Problem (1) is known as the DEA CCR model (Hadad, Keren & Hanani, 2013).
5. Previous studies (Literature Review)
This section presents a brief summary of the evolution of the studies on efficiency and
effectiveness in public transport. Chu, Fielding & Lamar (1992), analyzing the American bus
system, was one of the first works that considered both efficiency and effectiveness measures.
Efficiency was measured using a DEA model, with the output defined as the average return,
and input as the total cost of the activities related to the service. In turn, the effectiveness was
considered by the estimated number of unserved passengers. The authors observed that the
efficiency and effectiveness were correlated negatively and that a balance should be searched.
Pina & Torres (2001) compared the efficiency of the public and private transport sectors in
Spain in four different scenarios, but did not explicitly present effective measures for the quality
of service delivered to the user.
Karlaftis (2004), analyzing two hundred and fifty-nine American systems for five years, divided
them into three groups and three different models, using the same input set, but different output
sets: the first one, a model of efficiency, had its output defined as the total annual miles per
vehicle; the second, regarded as an effectiveness measure, considered as output the total number
of passengers transported per year; the third, a multi-output model, used as outputs both the total
annual miles per vehicle and the total number of passengers transported per year. The author
concluded that efficiency is positively related to effectiveness. However, it should be noted that
both indices are aligned with the interest of the service provider, since the higher number of
passengers, the higher would be the return.
According to Dyson at al. (2001), the data under consideration must be uniform, or the DMUs
must be performing the same activity with similar objectives. Besides, the inputs and outputs
must be identical for all the units, except for differences in value; the DMUs must be operating
in similar environments and under similar technical conditions, since the external environment
impacts on the performance of a unit.
Based on these considerations, Peña (2008) states that the "selected units should produce the
same goods and services using the same inputs." The above conditions aim to ensure that the
data for DMUs are reliable, and that possible extreme variations correspond to real situations,
no errors of judgment. Before the application of DEA, it is convenient to perform an exploratory
analysis.
Po, Guh & Yang (2009) proposed a new method for clustering DMUs, associating DEA with
the Ward’s algorithm by a linear production function. A similar procedure is used in the present
article. Amin & Emrouzejad (2011) showed that this approach may generate different groups
with different sizes, and allocations of DMUs in a group, which indicates that the problem of
grouping in DEA is still open.
Sampaio, Neto & Sampaio (2008) compared the efficiency of the public transport of seven
Brazilian cities to twelve European cities according to the following criteria: power partition
among the components of the administrative agency of the transport system and fare structure.
The authors concluded that only São Paulo, Brazil, was comparable to the best nine European
cities in terms of efficiency. We finally refer to the bibliography revision of Karlaftis, Gleason
& Barnum (2013), which presents a list with ninety-seven publications about the application of
DEA to the analysis of urban transit systems.
6. Application
This section outlines the application of our two-phase analysis methodology to a group of thirty
cities, with more than 42,000 passengers/day transported by urban, public bus transport systems.
All the data used can be found at http://www.brtdata.org. In the first phase, we use
parametric techniques to group units (cities) in homogeneous sets. In a second phase, we apply
DEA for evaluating the performance of each unit with respect to its peers in terms of efficiency
and effectiveness.
6.1. Aim of the study
The study addresses the important problem of defining guidelines for the public authorities,
which can assist them in the establishment balanced policies for both the service providers and
users of the urban public transport.
6.2. Data structuring
We consider thirty cities (DMUs), namelly Barranquilla (Colombia), Beijing (China), Belém
(Brazil), Belo Horizonte (Brazil), Bogotá (Colombia), Brasília (Brazil), Brisbane (Australia),
Cali (Colombia), Campinas (Brazil), Campo Grande (Brazil), Curitiba (Brazil), Fortaleza
(Brazil), Goiânia (Brazil), Guarulhos (Brazil), Guatemala (Guatemala), Guayaquil (Ecuador) ,
Istanbul, Johannesburg (South Africa), Lima (Peru), Manaus (Brazil), México (Mexico), New
York (USA), Ottawa (Canada), Porto Alegre (Brazil), Quito (Ecuador), Recife (Brazil), Rio de
Janeiro (Brazil), Salvador (Brazil), São Paulo (Brazil), Xiamen (China). Table 2 presents the
data of each DMU.
Fare - In a first analysis, from the point of view of the service provider, the higher the better;
for users, the smaller the better.
Fleet -The service provider seeks to minimize the size of the fleet for a given number of
passengers to be transported, while users prefer a fleet large enough to allow quality trips with
flexible timetables and destinations.
Demographic Density - It can be interesting for the service provider to operate in more
populated areas, which would provide higher returns with shorter routes.
Total passengers transported - Also aligned with the interest to the service provider, it may
be indifferent to the user, once the quality of service is guaranteed.
In Table 2, column 1 presents the DMUs, column 2 the total number of passengers carried per
day, column 3 the size of the fleet, column 4 the fare charged in each city, and column 5 the
demographic density.
Table 2 - Parameters considered in the analysis
City of
Passenger Fleet Fare D_Density City of
Barranquilla
115.000
39 0,62 3.650,00 Guayaquil
Beijing
305.000 187 0,00 1.006,00 Istanbul
Belem
100.000 120 0,70
661,80 Johannesburg
Belo Horizonte 1.047.374 3.066 0,96
524,70 Lima
Bogota
2.213.236 1.697 0,69 3.347,40 Manaus
Brasilia
301.000 139 0,78
491,60 Mexico
Brisbane
356.800 475 4,00
347,00 New York
Cali
471.361 722 0,59
327,00 Ottawa
Campinas
200.000 1.256 0,91
801,10 Porto Alegre
C_Grande
279.187 537 0,89
97,00 Quito
Curitiba
561.000 193 0,86
179,60 Recife
Fortaleza
286.777 1.804 0,62
625,30 Rio de Janeiro
Goiania
378.300 1.340 0,86
410,00 Salvador
Guarulhos
311.426 958 0,92 3.834,00 São Paulo
Guatemala
210.000 130 0,13 1.420,00 Xiamen
Passenger Fleet Fare D_Density
310.000 115 0,25 7.227,00
750.000 415 1,00 2.523,00
42.000 277 1,00 2.696,00
350.000 487 0,76 3.008,80
643.549 1.516 0,81
158,00
1.065.000 548 0,36 5.871,00
199.566 219 3,00
720,00
220.000 936 2,70
209,64
491.600 1.650 0,84
408,00
833.095 575 0,25
439,80
701.259 1.381 0,64 1.332,30
3.132.600 382 0,98 2.208,80
1.293.303 2.447 0,78
832,60
3.194.000 3.966 0,98 2.532,90
340.000 220 1,00 1.214,00
6.3. Determination of groups
6.3.1. Grouping according to Ward
The groups are determined using "proximity" as a decision criterion. With the data of Table 2,
the WARD’s method implemented in R (https://www.r-project.org) established four groups, as
shown in Figure 6; see also Table 5. Where Group 1 is formed by Bogota, Rio de Janeiro and
São Paulo, where the urban transport system carries more than 2,200,000 passengers/day. Group
2 consists of cities whose urban transport system carries 643,549 to 1,047,374, group 3 consists
of cities whose urban transport system carries 301,000 to 378,300, and the group 4 composed of
cities whose urban transport system carries 42,000 to 199,566 passengers. The later, two groups
are constituted by nine DMUs.
Figure 6 - Grouping of units by the WARD’s algorithm
6.3.2. Grouping by dominance of DMUs
DEA allow us to identify inefficient (E0<1) and efficient (E0=1) DMUs, as shown in Table 3,
column “Efficiency”. In column 3 to 9 are presented the multipliers obtained while solving
problem (1).
Table 3 – Summary of DEA dominance
City of
Barranquilla
Beijing
Belem
Belo Horizonte
Bogota
Brasilia
Brisbane
Cali
Campinas
C_Grande
Curitiba
Fortaleza
Goiania
Guarulhos
Guatemala
Guayaquil
Istanbul
Johannesburg
Lima
Manaus
Mexico
New York
Ottawa
Porto Alegre
Quito
Recife
Rio de Janeiro
Salvador
São Paulo
Xiamen
Score
Barranquilla Brisbane C_Grande Curitiba Manaus New York R_Janeiro
1,0000
1,0000 0,0000 0,0000 0,0000 0,0000
0,0000 0,0000
0,2010
0,6440 0,0000 0,0000 0,0804 0,0000
0,0000 0,4489
0,4674
0,0662 0,0000 0,0000 0,0000 0,0000
0,4739 0,0357
0,5461
0,0000 0,0000 0,0000 1,2864 1,8587
0,0000 0,0000
0,5037
0,0000 0,0000 0,0000 6,9040 0,0000
0,5724 0,9540
0,6425
0,0000 0,0000 0,0000 0,1404 0,0000
0,3307 0,1034
1,0000
0,0000 1,0000 0,0000 0,0000 0,0000
0,0000 0,0000
0,4435
0,0000 0,0000 0,0000 1,5785 0,2753
0,0000 0,0000
0,1284
0,0000 1,2792 0,6112 1,6588 0,0000
0,0000 0,0000
1,0000
0,0000 0,0000 1,0000 0,0000 0,0000
0,0000 0,0000
1,0000
0,0000 0,0000 0,0000 1,0000 0,0000
0,0000 0,0000
0,1515
0,0000 0,0000 2,5818 2,0776 0,0109
0,0000 0,0000
0,3057
0,0000 0,0000 1,9932 1,1823 0,0273
0,0000 0,0000
0,0980
0,0000 0,0000 0,0000 0,1781 0,0000
2,8163 0,8033
0,2249
0,2042 0,0000 0,0000 0,0000 0,0000
0,0567 0,2869
0,3856
0,6794 0,0000 0,0000 0,0000 0,0000
0,0000 0,2317
0,3409
0,1531 0,0000 0,0000 0,0000 0,0000
0,7339 0,6500
0,2606
0,5069 0,0000 0,0000 0,0000 0,0000
1,1746 0,0000
0,1749
0,2503 0,0000 0,0000 0,0000 0,0000
1,2162 0,5521
1,0000
0,0000 0,0000 0,0000 0,0000 1,0000
0,0000 0,0000
0,2391
0,1921 0,0000 0,0000 0,0000 0,0000
0,0000 1,4149
1,0000
0,0000 0,0000 0,0000 0,0000 0,0000
1,0000 0,0000
1,0000
0,0000 0,0000 0,0000 0,0000 0,0000
0,0000 0,0000
0,3630
0,0000 0,0000 0,7651 1,2831 0,6540
0,0000 0,0000
0,6014
0,0000 0,0000 0,0000 2,3819 0,0761
0,0000 0,0000
0,1725
0,0000 0,0000 0,0000 7,1051 0,0000
0,0000 0,0255
1,0000
0,0000 0,0000 0,0000 0,0000 0,0000
0,0000 1,0000
0,4663
0,0000 0,0000 0,0000 3,6215 1,1531
0,0000 0,0000
0,3968
0,0000 0,0000 0,0000 13,2900 0,9242
0,0000 0,0000
0,4551
0,0869 0,0000 0,0000 0,0000 0,0000
0,6511 0,1938
The first performance criterion was maximizing the fare and the number of passengers
transported, meaning that the higher fare and passengers transported, the higher the return for
the service provider. It was considered as inputs the fleet and the population density. Therefore,
this first criterion favors the service provider.
When the criterion is the service provider efficiency, Barranquilla, Brisbane, Campo Grande,
Curitiba, Manaus, New York, Ottawa, and Rio de Janeiro reach the maximum efficiency,
define the efficient frontier, and become reference units of the analyzed set. It can be inferred
that: (1) Barranquilla, New York and Rio de Janeiro dominate Beijing, Belém, Guatemala,
Guayaquil, Istanbul, and Xiamen; (2) Curitiba and Manaus dominate Belo Horizonte, Cali,
Porto Alegre, Quito, Salvador, and São Paulo; and so on. Ottawa, despite being efficient, only
dominate Campo Grande.
The second criterion was minimizing the fare and the number of passengers carried by the fleet;
that is, the lower the fare the better for the users. In addition, less passengers (users) per bus
would imply in a better quality of service. Using this criterion, Barranquilla, Beijing, Fortaleza,
Guayaquil, and Johannesburg determine the effectiveness frontier.
Table 4 – Efficiency and effectiveness
Score
Score
City of
EffectivenessEfficiency E2/E1
City of
EffectivenessEfficiency E2/E1
Barranquilla
1,0000 1,0000 1,0000 Guayaquil
1,0000
0,3856 0,3856
Beijing
1,0000 0,2010 0,2010 Istanbul
0,2161
0,3409 1,5775
Belem
0,2722 0,4674 1,7170 Johannesburg
1,0000
0,2606 0,2606
Belo Horizonte
0,9708 0,5461 0,5626 Lima
0,5020
0,1749 0,3484
Bogota
0,5381 0,5037 0,9360 Manaus
0,6117
1,0000 1,6348
Brasilia
0,1193 0,6425 5,3856 Mexico
0,6202
0,2391 0,3855
Brisbane
0,2074 1,0000 4,8216 New York
0,1697
1,0000 5,8928
Cali
0,0393 0,4435 11,2850 Ottawa
0,6614
1,0000 1,5119
Campinas
0,9930 0,1284 0,1293 Porto Alegre
0,6583
0,3630 0,5514
C_Grande
0,3051 1,0000 3,2776 Quito
0,4951
0,6014 1,2147
Curitiba
0,0772 1,0000 12,9534 Recife
0,6570
0,1725 0,2626
Fortaleza
1,0000 0,1515 0,1515 Rio de Janeiro
0,1136
1,0000 8,8028
Goiania
0,5629 0,3057 0,5431 Salvador
0,8636
0,4663 0,5399
Guarulhos
0,8135 0,0980 0,1204 São Paulo
0,8795
0,3968 0,4512
Guatemala
0,5428 0,2249 0,4143 Xiamen
0,2028
0,4551 2,2441
6.4. Analysis of performance of urban public transport
Table 4 and Figure 7 relate the efficiency and effectiveness criteria in these thirty cities.
Barranquilla provided the best result, being efficient and effective at the same time.
1.0000
Efficiency (EF1)
0.8000
C_Grande
Curitiba Brisbane
Manaus
New York
Rio de Janeiro
0.4000
Cali
Q4
Quito
Belo Horizonte
Salvador
Guayaquil
Porto Alegre SPaulo
Goiania
Mexico Johannesburg
Guatemala
Beijing
Recife Fortaleza
Lima
Guarulhos Campinas
Bogota
Belém
Xiamen
Istanbul
0.2000
0.0000
0.0000
Q2
Brasilia
0.6000
Ottawa Barranquilla
Q1
0.2000
0.4000
Q3
0.6000
0.8000
1.0000
Effectiveness (EF2)
Figure 7 – Efficiency versus effectiveness
The limits of the quadrants of Figure 7 were defined by the averages of efficiencies and
effectiveness. In Q1 are the cities of Belém, Cali, Istanbul, Lima, and Xiamen, which obtained
efficiency and effectiveness indices below average value. These cities should improve both
indices to reach quadrant Q4. In Q2 are the cities of Brasilia, Brisbane, Campo Grande,
Curitiba, New York, Quito, and Rio de Janeiro. These cities must keep their above-average
efficiencies and improve their effectiveness to reach Q4 quadrant. In Q3 are the cities of
Beijing, Bogota, Campinas, Fortaleza, Guatemala, Goiania, Guarulhos, Guayaquil,
Johannesburg, Mexico, Recife, Salvador, Sao Paulo, and Porto Alegre. These cities must keep
their above-average effectiveness and improve their efficiencies to reach quadrant Q4. In Q4 are
the cities of Barranquilla, Belo Horizonte, Manaus, and Ottawa. As these cities have aboveaverage efficiencies and effectiveness, they should work towards the point of excellence EF1 =
EF2 = 1.
Table 5 presents the cities in the four different groups determined by WARD’s algorithm. See
also Figure 6. In column “Initial” are the scores obtained by the thirty cities analyzed; in
“Cluster” are the four groups and respective scores within each group. Although Bogotá is
known as a good example of the urban transport system, it did not get good ratings because its
demography density and number of passengers per fleet are high. From the point of view of the
service provider (efficiency) the numbers of Bogotá are good, but from the point of view of the
users (effectiveness), they are bad. Another point is that in the case of the cluster; consider it as
belonging to the group 1, despite it being a cut off from this community.
Table 5 – Initial group versus cluster
Group
Group 1
Group 2
Group 3
Group4
Cluster
Initial
City of
Efficiency Rank Efficiency Rank
Bogota
0,4662
3
0,3212
20
Rio de Janeiro
1,0000
1
1,0000
1
São Paulo
0,8891
2
0,3967
16
Belo Horizonte
0,5460
6
0,5460
11
Cali
0,4435
7
0,4435
15
Curitiba
1,0000
1
1,0000
1
Istanbul
0,6218
4
0,3409
19
Manaus
1,0000
1
1,0000
1
Mexico
0,6687
3
0,2391
23
Porto Alegre
0,4195
8
0,3629
18
Quito
0,6013
5
0,6013
10
Recife
0,1747
9
0,1725
27
Beijing
0,7475
6
0,2102
25
Brasilia
1,0000
1
0,6426
9
Brisbane
1,0000
1
1,0000
1
C_Grande
1,0000
1
1,0000
1
Fortaleza
0,2644
9
0,1515
28
Goiania
0,4856
7
0,3058
21
Guayaquil
1,0000
1
0,3857
17
Lima
0,3283
8
0,1749
26
Xiamen
0,7683
5
0,4553
14
Barranquilla
1,0000
1
1,0000
1
Belem
0,7463
6
0,4675
12
Campinas
0,3129
8
0,1284
29
Guarulhos
0,3327
7
0,0980
30
Guatemala
1,0000
1
0,2249
24
Johannesburg
0,2606
9
0,2606
22
New York
1,0000
1
1,0000
1
Ottawa
1,0000
1
1,0000
1
Salvador
1,0000
1
0,4662
13
The dominant cities of each group are the references for other cities that did not reach efficiency
in the group.
(1) Group 1 - Bogota and Sao Paulo are dominated by Rio de Janeiro.
(2) Group 2 - Belo Horizonte, Cali, Quito and Porto Alegre are dominated by Curitiba and
Manaus; Istanbul, Mexico, and Recife are dominated by Curitiba.
(3) Group 3 - Beijing and Lima are dominated by Brasilia and Guayaquil; Xiamen is
dominated by Brasilia and Brisbane; Goiania and Fortaleza are dominated by Brasilia
and Campo Grande.
(4) Group 4 - Johannesburg is dominated by Barranquilla and New York; Belém and
Guarulhos are dominated by Guatemala and New York; Campinas is dominated by
New York and Ottawa; Salvador is dominated by New York.
It is worth nothing that each group has, at least, one dominant Brazilian city.
7. Conclusion
One of the problems faced by public authorities in urban public transport management is the
lack of references to be pursued. The two-phase method proposed enables us to determine
dominant and dominated cities in terms of efficiency and effectiveness indices, suggesting how
the dominated ones should follow the dominants to achieve the same level of performance.
The quality of life in the city is associated with the quality of urban public transport and one of
the duties of public authorities is to find a balance between efficiency (service provider) and
effectiveness (user) to ensure the continuity of the system. We believe that our work is a step in
this direction, since it allows determining what needs to be done to meet the user and the service
provider preferences, without favoring one or the other. The overall objective is to keep the
system evolving near the "sustainable return" line, Figure 7.
A natural extension of this work is the analysis of the locallity transport system, that is, to
consider routes: the number of buses, number of passengers, stops, and quality-of-service
parameters. Studies in this direction are currently being developed by the authors.
References
Amin, G. R. & Emrouzejad, A., (2011). “Some clarifications on the DEA clustering approach.”
European Journal of Operational Research, 215: 498-501.
Borger, B.; Kerstens, K. & Costa, A., (2002) Public Transit Performance: What Does One Learn
From Frontier Studies, Transport Reviews, vol. 22, no. 1 1-38.
Bussab, W. de O., Miazaki, E.S. & Andrade, D.F., (1990) Introdução à análise de
agrupamentos. In: Simpósio Nacional de Probabilidade e Estatística, São Paulo, Anais São
Paulo: ABE, p.106.
Cervero, R., (2013) Bus Rapid Transit (BRT): An Efficient and Competitive Mode of Public
Transport, Working Paper 2013-01 Berkeley Institute of Urban and Regional Development.
Charnes, A., Cooper, W. W., & Rhodes, E., (1978) Measuring the efficiency of decision making
units. European Journal of Operational Research, 2. 429-444.
Chu, X., Fielding, G. J., & Lamar, B. W., (1992) Measuring transit performance using data
envelopment analysis. Transportation Research Part A: Policy and Practice, 26A (3), 223-230.
Devarajan, S., & Shah, S., (2004) Making services work for India's poor, Economic and
Political Weekly, 907-919.
Drucker, P., (1997) An Introductory View of Management, Harper College Press, New York,
NY.
Dyson, R. G., Allen, R., Camanho, A. S., Podinovski, V. V., Sarrico, C. S., & Shale, E.A.,
(2001) Pitfalls and protocols in DEA, European Journal of Operational Research, 132, 245259.
Fancello, G., Uccheddu, B. Fadda, P., (2014) Data Envelopment Analysis (DEA) for urban
road system performance assessment, Social and Behavioral Sciences Was, 111, 780-789.
Hadad, Y., Keren, B., & Hanani, M. Z., (2013) Hybrid methods for ranking DMUs that combine
performance and improvement trend over successive periods, International Journal of Logistics
and Management, 16 (3), 269-287.
Ho, C.T. & Zhu, D.S., Performance measurement of Taiwan Commercial banks, International
Journal of productivity and performance Management, Vol. 53 No. 5, 425-433.
Karlaftis, M. G., (2004) A DEA approach for evaluating the efficiency and effectiveness of
urban transit systems. European Journal of Operational Research, 152, 354-364.
Karlaftis, M. G., Gleason, J. M., & Barnum, D, T., (2013) Bibliography of Urban Transit Data
Envelopment Analysis (DEA) Publications April 1. Retrieved from
SSRN: http://ssrn.com/abstract=1350583.
Kumar, S. & Gulat, R., (2010) Measuring efficiency effectiveness and performance of Indian
Public Sector Banks, International Journal of Productivity and Performance Management,
Vol. 59 (1), pp.51 – 74.
MacQueen, J., (1967) Some Methods for classification and analysis of multiattribute instances,
Proceedings of the Fifty Berkeley Symposium on Mathematics, Statistics and Probability,
volume 1, pages 281-296.
Morini, C., Barassa, E., Maurice, M. P. G., Moretti, A. C. Porto, P. C. S., (2014) Customs of
Brazil and competitiveness: a comparison in terms of relative efficiency, GEPROS. Production
management, operations and systems, Bauru, Year 9, no. 3, jul-Sep, p. 1-13.
Mouzas, S., (2006) "Efficiency versus effectiveness in business networks", Journal of Business
Research, vol. 10/11 Us 59, pp. 1124-32.
Peña, C. R., (2008) Um modelo de avaliação da eficiência da administração pública através do
método análise envoltória de dados (DEA), Revista de Administração Contemporânea, v.12,
n.1.
Pina, V. & Torres, L., (2001) Analysis of the efficiency of local government services delivery:
an application to urban public transport, Transportation Research, Part A 35, p. 929-944.
Po, R.W., Guh, Y.Y. & Yang, M.S., (2009) A new clustering approach using data envelopment
analysis. European Journal of Operational Research, 199, 276–284.
R Software – Retrieved from https://www.r-project.org.
Raphael, G., (2013) Measuring efficiency, effectiveness and performance of Tanzanian
commercial banks: A two stage analysis. European Journal of Business and Management, vol.
5, no. 7.
Sampaio, B. R., Neto, O. L. & Sampaio, Y., (2008) Efficiency analysis of public transport
systems: Lessons for institutional planning, Transportation Research Part A: Policy and
Practice, Volume 42, Issue 3, pp 445-454.
Soliman, M., Siluk, J. C. M., Neuenfeldt JR, A. L. , Married, F. L. & Paris, S. R., (2014)
Modeling for Evaluating the Technical Efficiency of University Units, GEPROS. Production
management, operations and systems, Bauru, Year 9, no. 1, jan-mar, p. 69-83.