European Journal of Sport Science, vol. 2, issue 2
Performance Taxonomy / 1
©2002 by Human Kinetics Publishers and the European College of Sport Science
Development of a Taxonomy of Performance
for National Sport Organizations
Emmanuel Bayle and Alberto Madella
The measurement of performance of a voluntary organization challenges management theorists and practitioners. What are the effectiveness and efficiency
criteria to evaluate performance of these organizations? To overcome the limitations of current methods of performance measurement, we propose a new
performance measurement technique in a taxonomic perspective that associates qualitative and quantitative indicators thanks to a contractual approach.
This approach establishes a typology of 6 performance profiles of 40 NSO.
Statistical correlations between performance dimensions are also analyzed.
Statistical analysis shows that dimensions of performance are significantly
correlated. Economic and financial performance is correlated more with promotional performance than with the number of members.
Key Words: National sport organisation, performance, effectiveness, measure,
control.
Key Points:
• A definition of performance for national sport organizations is proposed: six dimensions have been identified: institutional (membership and elite sport results),
social external, organizational, promotional, financial, and internal social performance.
• A specific technique of measurement to compare and classify the performance of
40 NSO on each dimension is proposed.
• Dimensions of performance are significantly related. Economic and financial performance is correlated with promotional performance but less with the number of
members.
Introduction
Organizational performance is recognized by theorists and practitioners as a multidimensional concept for all kinds of organizations: private, public, or non profit.
This concept, which is already difficult to apply in general to service organizations
and companies, is even harder to be operationalized when voluntary organizations
are concerned. This occurs essentially because of the too generic and broad goals of
this kind of organization. The evaluation of the programs and actions of non-profit
E. Bayle <[email protected]>, Maître de conférences Université de Lyon 1 CRIS
(EA 647) UFR STAPS 27/29 boulevard du 11novembre 1918 69 622 Villeurbanne cedex,
France. A. Madella <[email protected]>, Docente di Organizzazione degli Organismi Sportivi,
Università di Firenze, ISEF, Viuzzo di Gattaia 9, 51025 Firenze, Italia.
1
2 / BayleandMadella
organizations is complicated by the multiple goals, multiple constituencies, and
partial market isolation characterizing most of these associations. The purpose of
this article is to define a concept of performance1 and a method for its measurement
to determine performance profiles for national (French) sport federations. These
organizations, in charge of a public mission (i.e., organization, promotion, and
development of the sport practice), govern in France a relevant area of social interest
and activity (170,000 clubs and more than 14 million members).
I. A Framework to Study NSO Performance
We will analyze first a definition of performance suitable for the measurement of the
global organizational performance in general and for the national sport organizations (NSOs) in particular. Next, we will propose new measurement criteria capable
of producing a typology of performance profiles of 40 French NSOs. We will show
the specific and contextual nature of the organizational performance concept, and
we will focus finally on the variety of the theoretical measurements available in the
literature.
1. The Concept of Performance for NSO
The concept of performance is generally understood as a combination of the popular
criteria of efficiency and effectiveness (Figure 1). Effectiveness is traditionally
defined as the capacity to achieve the institutional goals. Efficiency compares the
means used and the real production without examining the satisfaction of the user.
For private companies, the financial value and assets represent the basic
synthetic measure of performance. For NSOs, efficiency may not always be particularly relevant2, and effectiveness is a complex construction with an external dimension (external social performance). This complexity calls for a reflection on the
most appropriate measurement methods. Like other voluntary organizations, some
NSOs may have a commercial purpose to increase their budget. The fact that in
principle profit cannot be distributed is often irrelevant in practice and not the main
obstacle to the development of a performance appraisal system.
The performance of this kind of organization must first be related to their
capacity to obtain and use human, financial, and material means to reach institutional and ideological goals. This concept reflects the importance of the judgments
decided by the organization’s multiple constituencies (i.e., individuals or groups of
individuals) in aid of a values system, and of services, outcomes, or effects they
Figure 1 — The two components of performance: effectiveness and efficiency.
Performance Taxonomy / 3
Figure 2 — Specific elements of performance for voluntary organizations.
expect from the organization as a result of its activity. This implies that the subjective
perception of the performance is always a relevant component of its measurement,
and this is clearly an additional source of methodological problems. For this reason,
the measurement of performance of a voluntary organization must necessary combine objective and subjective measurements to give a broader and realistic vision of
the global performance. Figure 2 summarizes the main specific characteristics of the
performance of a voluntary organization.
2. Theoretical Methods of Performance Measures
Various classical models with methodological implications for the definition and
measurement of performance may be identified in the literature: (a) the goal achievement model, (b) the open system model, (c) the internal process model, and (d) the
strategic constituencies model. Two other holistic models can also be added: (e)
competing values model and (f) the four dimensions model proposed by Morin.
Table 1 indicates the main characteristics and limitations of each of these
models with reference to the specific aspects of NSOs. A general evaluation of the
possibility of operationalizing the different methods in this research area is also
provided.
Table 2 illustrates the most popular applications of most of the above mentioned theoretical approaches to performance assessment of sport organizations.
Only research directly aimed at measuring the performance of sports organizations
has been analyzed. We specify in Table 2 the measurement method, the sample used
in the study, the most significant results, the contribution for the further development of the field, and the main limitations3 of the research regarding our aim.
For all the cited studies, specific problems of validity, reliability, and
operationalization may be identified even if the indicators and results found are very
interesting and useful to further methodological refinement. Of course, the methodology must be evaluated based on the goal of each individual study, not just generally. In this case, our goal is to establish a typology of performance profiles of NSOs
and to explain their performance differences on the basis of a global performance
measurement of each NSO4. This goal emphasizes the comparative aspect of the
4 / BayleandMadella
Table 1 Characteristics and Limitations of the Main Theoretical Models
of Organizational Performance
Definition
Organizations to
(an organization is which the model
effective when)
can be applied
Limitations of the
model due to the
specificity of NSO
Goal attainment
It accomplishes its Goals are clear,
stated goals
time-paced,
measurable.
Goals are often
vague and
unrealistic.
—
System resource
It acquires the
resources needed
Clear relation
between inputs
and outputs.
Some resources
come from the
trusteeship and
are annually
renewable.
—
Internal process
It presents no
internal strain,
with smooth,
internal functioning.
Clear relation
This relation is not
between organiza- so clear as for a
tional process
private company.
and the primary
goal.
—
Strategic
constituencies
All strategic
constituencies
have a minimum
degree of satisfaction.
Constituencies
Applicable to the
have powerful
analysis of the
influence on
influence of
the organization
competent Ministry;
(as in times of little but difficult to be
organizational
operationalized—
slack) and it must weak validity.
respond to
demands.
+
Competing values The evaluation of
the organization
in four areas
matches constituent preferences
The organization Difficulty of realizahas no clear view tion for non-profit
of its own priori- organizations.
ties,or show a
quickchange in the
criteria over time.
—
Quadridimensional model
Performance of
Legitimacy
the organization is measure.
measured by a
subjective and
objective approach.
—
Model
Systemic evaluation of performance
Note. Adapted from K.S. Cameron (4).
Operational
level
Strategic constituencies
model
Internal process model
(empirical and quantitative study of the
concept of performance
for NSGB)
System resource model (Questionnaires sent
(relation between ento a sample of 835
vironment and effective- club directors)
ness of a club)
Vail (16)
Chelladurai
et al. (5)
Koski (8)
Study of the relation between structure and
effectiveness
Results
Questionnaire of 30
indicators (150 directors of NSGB of 48
Canadian NSGB)
Group differences in
the perceived importanceof selected effectiveness criteria: growth
and finances (internal
groupsperceive them
more crucial)
Definition of
indicators for NSGB
Relevance
Five dimensions of effectiveness of a club; interdimension correlations;
variation of the effectiveness cycle
Systemic approach to
variables affecting
performance
Proposition of a model of
Pertinence of the
six dimensions
methodology
Critical dimensions:
throughput process,
human resources factor
and results of elite programs
Top level results and sport
for all not related:
Five strategic groups Six dimensions
(140 questionnaires (36 performance criteria)
for 33 NSOs)
of performance: adaptability,
communication, finance,
growth, human resources,
and organizational planning
Quantitative study
(sample unknown)
Goal attainment model;
system resource model
Frisby (7)
Sample
Model
Measurement Methods and Approaches to the Performance of Sport Organisations
Authors
Table 2
(continued)
Synchronic measure of
variables; specificity
for clubs (not applicable to a NSO)
Measure of the quality
of functioning more the
results: specific to the
Canadian context; synchronic measure of
input and output
indicators
Key indicators as
sport results, number of
members are not
considered
Variables of performance measurement
not take into account
Limits
Performance Taxonomy / 5
Multidimensional approach (goal attainment;
system resource; strategic constituencies)
Madella (16)
Definition of several
indicators (social
climate, social image,
technical and economical, and sport performances)
Measure of legitimacy
of the organisation for
the main strategic constituencies
6 dimensions of
Combination of a
performance: finances,
theoretical and practical
external communicainterest
tion; internal communication; sport
results; service quality
and production; logistics
and process factors.
Indicators of environmental condition and other Input
variables were collected
Combination of
official statistics and
other objective indicators andquantitative
evaluation by the
stakeholders
Internal process model
(relations between
management of
human resources and
effectiveness)
H. Mahé de
Boislandelle
(11)
Six strategic constituencies groups;
study on 20 Greek
NSO
Five dimensions of
effectiveness (board
and external relations
stability; athletes’ care;
internal procedures; longterm planning; contribution
of sport sciences) 33 indicators. Athletes, technical
managers are less satisfied
than members of the board
about organizational performance.
Qualitative approach Proposition of indicators
proposed for profes- (4 variables for managesional sport clubs
ment of human resources
and four indirect effects)
(continued)
Papadimitriou Strategic constituencies
(13)
model
Table 2
Difficulty in adapting
the measurement
system to the political
change of priorities
No empirical
application
Reliability and validity
problem of this method;
difficult operationalization
6 / BayleandMadella
Performance Taxonomy / 7
analysis and influences the specific measurement method adopted.
II. A Specific Method to Build a NSO Performance Taxonomy
Our method is based on a “contractual” approach that allows one to identify dimensions and indicators of performance from an inductive and qualitative perspective.
We present the data treatment that permits us to propose our taxonomy. We finally
study statistical correlations between performance dimensions.
1. Method: A Contractual Approach to Measure
Sport Organization Performance
First, we will analyze and determine the expectations of the different constituencies
of NSOs. This step is necessary in order to set realistic and valid performance
criteria. This approach permits us to show the discrepancy between the particular
expectations of selected constituencies and the actual results obtained. In this way,
we can advance our understanding of organizational dysfunction. To this purpose,
we have chosen the 6-step model developed by Spriggs (15) for voluntary organizations (Table 3).
We have already emphasized the multiplicity of expectations of different
stakeholders. The expectations of the different stakeholders of a NSO (Figure 3)
permit us to identify implicit and explicit criteria of performance that can be used for
its evaluation.
All the listed shareholders have a strong and stable interest across the whole
lifecycle of the organization (growth, maturation, decline). The more the NSO
depends on a particular actor for funding (members, ministry, sponsors, etc.), the
more important it is to satisfy the expectations of the specific funding organization.
Furthermore, network strategy becomes more and more complex with the growth of
the voluntary organization. For each stakeholder, precise expectations and indicators have then to be reconstructed on the basis of the literature and interviews with
stakeholders of the NSO.
As a specific example, Table 4 summarizes the expectations of the French
Youth and Sport Ministry, one of the most fundamental stakeholder of NSOs in
France.
The expectations of stakeholders permit us to determine seven “empirical”
modalities of organizational performance, each requiring a specific evaluation. The
characteristics of these modalities of performance evaluation are summarized in
Table 5.
Three of the methods reported have a legal basis—a political, legal, or conventional evaluation carried out by the national trusteeship (ministry). Three other
methods of evaluation are more informal but can affect the others. Any one of these
principles and evaluation methods may guarantee a global and impartial performance evaluation (9).
In spite of this, an objective evaluation of performance cannot be wholly
obtained because of several conditions:
• the impossibility of comparing results with competitors (other NSOs of different sports with different competition systems and sport performance appraisal methods);
Interviews with
stakeholders and
expertsof the
sector
Methods
Literature and
hierarchy of
performance
dimensions
provided by
managers
0
1
Identification
Description
of groups and
of the nature
their expectations of performance
3
Definition of
the evaluation
Literature and
hierarchy of
performance dimensions provided
by managers
2
Definition
of dimensions
Spriggs (15) Approach to a Voluntary Organisation
Stages
Table 3
5
Indicators
construction
6
Verification
of the relevance
of the approach
by sector experts
Document anaExpert validation Opinion of
lyses;questionof the indicators experts
naire; financial
accounts;analysis
of obtained results
4
Research
methods
8 / BayleandMadella
Performance Taxonomy / 9
Figure 3 — Stakeholders expectations with respect to a NSO.
Table 4
Expectations of French Youth and Sport Ministry Regarding a NSO
Main expectations
Indicators
Financial equilibrium
Accounts
Financial autonomy
Ratio of NSO financial dependence on Y.S.M
National representation in
major international competitions
Number of medals won in the European and
World Championship for the different teams
Employment creation in sport
Number of created jobs (sport teachers...in the
sport)
Economic impact and
development of sport
Increase of turnover in the national sport
industry; direct and indirect economic impact
Institutional development of
sport practice and quality of the
national network
Increase of members, clubs active at the national,
regional and local level
Development of the teaching
framework
Number of active volunteers and professionals
with teaching and coaching responsibilities
Reinforcement of the legitimacy
of the political power
Mediatization of the sport and results of athletes
Respect of legal obligations
Number of litigations who justify the intervention
of youth and sport ministry; financial and
accountant procedures
Financial transparency
Accuracy of financial statements, acceptation of
procedures of controls
Annual
Evaluation period
Annual (all 4
years for the
elections)
National
trusteeship of
the (Youth
and sport
ministry)
Qualitative and Quantitative
quantitative
Legal and
conventional
Organism in charge General
of the evaluation
assembly
(members of
the NSO)
Evaluation criteria
Political
Annual
Chartered
accountant
Quantitative
Legal
(accountant
and financial)
and contractual
Variable
International
federation
Quantitative
International
system
(international
federation)
Different Performance Evaluation Methods Applied to NSOs
Evaluation
methods
Table 5
Management
Variable
(depending on
terms of conractst and
interests of
individuals)
Sponsors,
multisportfederations;
employees
of the NSO
Variable
(stronger
during crisis
or just before
major sport
events…)
General media
and sport
media
Variable
(different in the
organizational
sectors)
Management system
(volunteers and
professional
managers)
Qualitative and Qualitative and Qualitative and
quantitative
quantitative
quantitative
Conventional
and contractual Media
10 / BayleandMadella
Institutional
Obtaining the best
sport results
(national teams);
develop number
of members
High level results
and number of
members
Quantitative
(qualitative)
Objectives
Variables
measured
Measurement
methods
Qualitative
Degree of
satisfaction
of actors
Improving social
atmosphere and
implication of
the actors
Social internal
Qualitative
Social legitimacy
and effects of
NSO activities
on the society
Contributing to
the achievement
of institutional
goal and a better
functioning of
the society
Social external
Dimensions of Performance and Tools for Their Measurement
Dimensions
Table 6
Increasing sport’s
media coverage
among sport
practitioners and
public opinion
Promotional
Quantitative
Quantitative and
qualitative
Access to finan- Notoriety and
cial resources,
image
resources diversification, capacity
of self-financing
Obtaining
resources necessary to achieve
goals; managing
properly financial
dependence on
public authorities
Economic
and financial
Qualitative
Quality of functioning
and organizational
reactivity
Running the headquarters of the
federal system
efficiently and
matching the mission
statement and the
environment demands
Organisational
(strict)
Performance Taxonomy / 11
12 / BayleandMadella
• the internal evaluation is often grounded on subjective, emotional, and nonmeasurable criteria;
• the use of independent experts is set by researchers as the condition of a
complete and real evaluation of a nonprofit private or public organization; this
condition was not fulfilled for NSOs that we studied.
It appears that the modality of performance evaluation is political at the
member level, with the youth and sport ministry, International Federation, or the
media.
In order to conduct an empirical study of performance according to the specific conceptual framework discussed in the previous section, interviews with volunteers and professional managers were carried out, and a specific questionnaire
was administered. In agreement with what was already reported by Madella for Italy
(10), “differences in performance conceptualisation among the stakeholders (evaluated through qualitative interviews) were quite limited and allowed anyhow the
construction of a shared model”.
2. Dimensions and Performance Indicators
Performance indicators have been gathered together by experts around six dimensions: institutional (membership and elite sports results), organizational, financial,
promotional, social internal, and social external.
Performance evaluation on these six dimensions permits a multidimensional
representation of NSO performance. Table 6 lists the six dimensions of performance
and tools for their measurement.
For NSOs, the attainment of the institutional goals are usually more complex
than for a other voluntary organizations like clubs or leisure associations. It involves
in fact the integrated action and intervention of the NSOs, clubs, and local authorities (regional and departmental governing bodies).
The major problem with measuring economic and financial performance is
isolating the economic performance of a NSO from the global economic performance of a nation’s entire federal system. Moreover, the analysis of performance of
each sport (and therefore the possibility of any kind of comparison) must take into
account the fact that some sports are managed by large national organizations (e.g.,
National Football League, French Football Federation) and that, conversely, an
individual organization may manage all sport activities within a single structure.
(This is the case with the French Tennis Federation, which manages for example the
Roland Garros and Bercy tournaments.)
Social external performance may be understood as the large social and economic contribution of the activities of the NSO to society (e.g., health, education,
and well-being of citizens; economic development). The difficulty in this case is in
measuring the social and economic impact of the NSO’s activities. How is value
determined for an Olympic medal or a world champion title? How does sport practice affect community health? How are sport values transmitted to NSO members,
and how does society benefit?
A further element of complexity is that societal performance is too a performance of legitimacy (subjective evaluation) with an ideologic dimension.
Aware of previously mentioned methodological and substantial issues, for
each of the six selected performance dimensions, we constructed a set of items
through interviews and questionnaires, taking also into account the results of former
Performance Taxonomy / 13
investigations and the expectations of the stakeholders. After a pretest, our questionnaire was sent to 53 directors in the National Sport Federation in France.
The definitive sample is composed of 40 federations (22 Olympic federations
and 18 non-Olympic federations), which indicates a very satisfactory reply rate of
75.5%. All the biggest federations are represented in the study. Each director had to
evaluate the performance of his or her federation in 1997 and also in the period of
1992–93. We adopted a longitudinal approach to measure performance. The period
between 1997 and 1992–93 is a longer cycle than the traditional Olympic cycle (that
is, the typical period during which change can become manifest: presidents withdraw, new financial partners are found, etc.).
In the questionnaire, we adopted a scale of measure with 7 modalities, which
permits the possibility of a neutral answer.
A specific questionnaire was administrated to a NSO expert (usually the
director of a NSO at the Youth and Sport Ministry) in order to measure social
external performance in the period considered. Two specific questionnaires were
administered to experts in order to measure two specific performances: high level
performance of the national teams (evaluated by the Director of Olympic Preparation) and evolution of the number of NSO members (evaluated by a NSO director at
the Youth and Sport Ministry).
Subjective data were collected through four questionnaires to measure relative perceived performance. Objective indicators permit the measuring of absolute
performance.
For the objective data, we adopted a 10-year measurement period, which
allowed a longitudinal analysis of the performance. All indicators were added together in order to evaluate each dimension.
Table 7
Economic and Financial Performance Indicators
Variable to measure
Indicator
Information source
Capacity to manage
financial dependence
Dependence rate on Youth
and Sport Ministry;
evolution of the
dependencerate
Information from accountancy
of NSO and Y.S.M.
Capacity to obtain
resources
Total of annual accounts;
evolution of total annual
accounts in proportion to
the number of members
between 1989 and 1997
Accountancy of NSO
Capacity to invest;
financial health and
equilibrium
Financial ratio in 1996
Accountancy of NSO
Capacity to generate
resources in the
federal system
Capacity to generate
subsidies from the local
authorities
Financial study of local
authorities funding
14 / BayleandMadella
The measure of economic and financial performance, for example, was attained through 7 objective indicators and 7 subjective indicators (Table 7).
To measure social and economic contribution of each NSO to society, we
decided not to use common economic methods (cost advantages or economic impact studies)—see, for example, Andreff (1) and Barget et Gouguet (2)—since
under the specific conditions, it was not possible for 40 NSOs to collect the information required to carry out such studies5. Due to these methodological difficulties, we
decided to evaluate social external contribution of each NSO by a qualitative measure—that is, the degree of legitimacy, assessed by an expert of the Youth and Sport
Ministry on the basis of the results and activities of NSOs. A NSO director at the
Ministry of Youth and Sport was asked to classify each NSO on three dimensions:
• the way each NSO accomplished its public service mission;
• the effectiveness of the communication of the ethical message (e.g., values,
sport spirit) to its members;
• the social and economic impact of the activities of a NSO to civil society.
Within this framework, we distinguished the organizational performance of
the national headquarters from that achieved by the NSO territorial system (regional
governing bodies and clubs). We evaluated the organizational performance through
the capacity of the NSO to meet, from an organizational point of view, the expectations of the sponsors, the Youth and Sport Ministry, and its members. We took into
account the quality of the flow of information between different services of the
federal headquarters, the capacity of NSO employees to solve technical problems
(administrative, juridical, etc.), the power of managers to create projects, and the
overall capacity of the employees to carry out their operations.
Territorial organizational performance was evaluated through the quality of
policies of the regional and departmental sport governing bodies. Twenty-two items
for organizational performance of the headquarters and six for the performance of
the system had been selected.
As for organizational performance, experts were asked to evaluate social
internal performance in terms of the quality of the relationships between and among
volunteers and staff employees.
Promotional performance was judged by the director based on the evolution
of media exposure (notoriety of NSO, a subjective indicator), and through an objective indicator: TV coverage between 1993 and 1997. Promotional performance was
judged too by the opinion of the NSO’s director about the impact of the NSO’s
communication to the members (in terms of all the means of communication and
promotion).
Fifty-five subjective indicators and 10 objective indicators were finally used
to measure the performance. The main limitation of our method is the absence of
objective measurement indicators for three of the six dimensions: organizational
performance, internal social performance, and social external performance.
As mentioned earlier, our goal was to set up a qualitative approach to indicators of NSGB performance that is able to provide a global measure of performance
typology of French national federations and highlight the existence of different
performance profiles through a typological approach.
3. Data Treatment to Build the Taxonomy
Performance Taxonomy / 15
In this specific case, we made a methodological choice not to balance the importance of the different indicators for the statistical analysis.
Mintzberg has pointed out that the relationship between explicative variables
and effectiveness cannot be analyzed simultaneously at a specific time (12, p. 209).
We tried to avoid this limit by studying performance over the course of a decade
(1988–1997) for objectives measures and for 5 years (1992–1997) for subjective
data.
We established a performance index built on the average scores of the indicators for each dimension. The overall performance can be described through a representation with 7 axes, since institutional performance was differentiated in two subdimensions: membership and elite sport results.
Optimal NSO performance is indicated by the highest score for each axis and
the homogeneity of the scores (see Figure 5 for an example).
Performance profiles for the 40 NSO were established through a hierarchical
classification method. Parametric statistics were used, since our data had interval
characteristics (7-point scale), and the sample was larger than 30 cases. Evrard et al.
(6) considered these two conditions sufficient to justify the utilization of parametric
tests.
We chose the distance of Ward, which gives a more complete measure than
center of gravity and distance average because it takes into account the dispersion of
points of each class.
Thanks to these methodological choices, typology respects a certain number
of validity criteria. The results obtained on the sample (40 of 53 federations) appear
valid given the size of the sample. In agreement with the specific methodological
frame of reference, six performance profiles were derived.
The major interest of this hierarchical typology is that we can visually represent proximities between individuals and classes. Six groups of federations may be
identified through the dendrogram (Figure 4) produced with the statistical analysis
carried out with SPSS/Win.
After the determination of scores, averages, and rankings of performance for
each NSO, six denominations were used to qualify their performance profiles: the
mighty, the effective, the dilemma, the atypical, the defective, and the problematic.
Class 1: The Mighty. Five federations (football, tennis, rugby XV, track and
field, cycling) belong to this class, with homogenous and high scores across all
dimensions. These sports have good media exposition, which is the main characteristic distinguishing class 1 federations. These federations have the most important
financial resources. However, it must be noted that one of the problems with our
method is that the index of promotional performance is affected positively by nonorganized events that do not properly belong to a sport federation (as is the case of
the Tour de France and the French Cycling Federation). These events do not provide
direct benefit to the federation but affect their performance index. This fact explains
why the global performance and economic potential of some sports are weaker (e.g.,
cycling and track and field).
Class 2: The Effective. This cluster includes the large majority of Olympic and
traditional federations and is particularly efficient for high-level sports. For these
federations, the majority of the indexes of all dimensions of performance present
16 / BayleandMadella
Figure 4 — Hierarchical analysis, dendrogram (Ward method).
very high scores, save for one problematic performance (generally promotional
performance or, less frequently, financial performance).
Class 3: The Dilemmas. These Olympic and non-Olympic federations often
have two dimensions of weak performance and average scores for the rest. The two
problematical performance dimensions are quite often the promotional performance
and high-level sport results performance. In some cases, one of the performance
dimensions can be extremely weak. This group of federations is qualified as Dilemma Federations because they do not develop properly regarding their potential
and financial resources. Institutional performance can be unsatisfactory either in the
high sport results (e.g., basketball and golf) or in the evolution of the number of
members (e.g., ski).
Class 4: The Atypical. This class includes federations presenting heterogeneous
performance profiles compared with other classes of federations. These federations
Performance Taxonomy / 17
are not necessarily ineffective but show a performance level much weaker than the
others. This group is made up of federations with performances that are globally
very different. Some of them are very close to class 2 and others to class 5.
Class 5: The Defective. These four federations appear to be in great difficulty.
They have weak scores on all 7 dimensions of performance and particularly a very
weak societal performance. Their financial situation is generally precarious. The
French Weightlifting Federation, which filed for bankruptcy in 1998, is an example
of a class 5 federation.
Class 6: The Problematic. These federations are in difficulty in at least two
dimensions of performance, notably economic and financial performance. This can
Figure 5 — “Radars” of the six performance profiles. Orga: score on organizational
performance; Eco fi: score on economic and financial performance; Societale: score on
social external performance; Staut licen: score on number of members P; Socia Int:
score on internal social performance; Staut eli: score on sport results (in high level
competitions); Promo: score on promotional performance.
18 / BayleandMadella
be explained through the fact that they were, at the time of the study, in a phase of
reconstruction.
“Radar” diagrams in Figure 5, provide visual examples of each typology of
federations.
In the typology that we proposed, possible evolutions can appear regarding
the actual context of certain federations.
4. Statistical Correlations Between Performance Dimensions
Our statistical correlation analysis shows that dimensions of performance are significantly related (Figure 6). Economic and financial performance seems to be
related to promotional performance but less so with the number of members. Social
external performance is correlated with organizational performance and economic
and financial performance.
In order to have a clearer view of the relative importance of the components of
global performance, the linear correlation between the performance indexes calculated for each dimension and the general performance index (each time not including the specific index referring to the dimension analyzed) was calculated. As
shown in Figure 7, the global performance index (not incorporating the value of the
economic and financial performance index) was strongly correlated with the economic and financial performance index.
Some areas of performance show relative independence with respect to the
economic dimension. This is for example the case of top level sport performance,
Figure 6 — Correlation between the seven dimensions of performance. Orga: organizational
performance; Eco fi: economic and financial performance; Societale: score on societal performance; Licen: performance on number of members; Int: internal social performance; Eli:
sport results performance (in high level competitions); Promo: promotionnal performance.
Performance Taxonomy / 19
that can probably be partially explained through the level technical know-how and
the historical and social capital available in the federations. This is the reason why
performance indicators both economic and financial, are not always related to this
dimension of performance.
Figure 8 shows a significant correlation between the evolution of the number
of members between 1988 and 1997 (“Evoli”) and elite results in 1997 (“Eli” 1997).
Such a relation is logically explained, since top-level sport (particularly team sport)
requires a certain number of practitioners and therefore a sizeable number of federation members.
Sport results have been mentioned often as possible factors in the evolution in
the number of members in a federation, but this opposite relation cannot be demonstrate through data analysis, maybe because the Coubertinian model (“high level
sport attracts mass sport”) is today not the dominant model.
Figure 7 — Correlation between economic and financial performance and global performance (not including the economic and financial performance index value). Soincom:
global performance on all dimensions; Eco fi: economic and financial performance.
Figure 8 — Correlation between the change in the number of members between 1988
and 1997 and results of high-level sports in 1997.
20 / BayleandMadella
The possibility of establishing causal relations between dimensions of performance is only partial, however, given the limitations associated with our methodological choices. For this reason, this analysis requires further longitudinal study to
confirm our performance dimensions and indicators.
Conclusion
The typology and methodology of our investigation overcome some of the limitations of previous approaches to the measurement of organizational performance in
sport. The advantage of our method is that it improves the possibility of combining
output measures to take into account the multidimensionality of organizational
performance.
Another main advantage is the possibility of comparing one NSO’s performance with another. As with all typologies, though, the final outcome of any similar
approach will always be reductive due to the indicators and methodological implications concerning access to data or other statistical analyses. A longitudinal study
to assess the performance of each NSO is another interesting research project for the
future. Such a study would permit more indicators specific to the NSO (see 9).
References
1. Andreff W. 1991. Les effets d’entraînements des jeux olympiques d’Albertville, retombées
socio-économiques et innovations dans le domaine du sport en région Rhône-Alpes.
Programme PPSH 15.
2. Barget E, Gouguet J-J. 2000. L’impact économique du spectacle sportif: analyse critique
de la littérature. Reflets et perspectives de la vie économique. Numéro spécial, Sport et
mondialisation, quel enjeu pour le XXIème siècle? Tome XXXIX:2-3, De Boeck
Université. p. 17-34.
3. Bayle E. 1999. Management et performance des organisations à but non lucratif le cas
des fédérations sportives nationales [dissertation]. Université de Limoges.
4. Cameron KS. 1984. The effectiveness of ineffectiveness. In: Straw BM, Cummings LL,
editors. Research in organizational behavior: an annual series of analytical essays and
critical reviews, Volume 6. Greenwich: S.T. J.A.I.Press. p. 235-85
5. Chelladurai P, Szyslo M, Haggerty T. 1987. Systems based dimensions of effectiveness:
the case of the national sport organizations. Canadian Journal of Sport Sciences 12:11119.
6. Evrard Y, Pras B, Roux E. 1997. Market, 2nd ed. Nathan.
7. Frisby W. 1986. The organizational structure and effectiveness of voluntary organization: the case of Canadian sport governing bodies. Journal of Park and Recreation Administration 3:61-74.
8. Koski P. 1995. Organizational effectiveness of finish sports clubs. Journal of Sport
Management 9:48-59.
9. Le Duff R, Papillon JC. 1997. Gestion du non marchand. Encyclopédie de gestion. p.
1605-20.
10. Madella A. 1998. La performance di successo delle organizzazioni—spunti di riflessione
per gestire efficacemente le societa di atletica leggera. Atleticastudi 1:2-3.
11. Mahé de Boislandelle H. 1997. Les variables influençant les performances technicoéconomiques et sportives. In: Pigeassou C, Garrabos C, editors. Management des
Performance Taxonomy / 21
organisations de services sportifs, P.U.F., pratiques corporelles. p. 154-74.
12. Mintzberg H. 1986. Le pouvoir dans les organizations. Les Editions d’organisation.
13. Papadimitriou D. How well Greek national sport organizations do? Perceptions of organizational effectiveness based on the multiple constituency approach. Proceedings of the
Second European Congress on Sport Management, Florence, September-October 1994.
p. 505-18.
14. Revat R, Madignier V. 1996. Comment mesurer l’impact des projets d’intérêt général
sur l’opinion publique? In: Le Duff R, Rigal J-J, editors. Premières rencontres ville et
management. Pau: Presses Université de Pau. p. 395-406.
15. Spriggs MT. 1994. A framework for more valid measures of channel member performance. Journal of Retailing 70:327-43.
16. Vail SE. 1986. Organizational effectiveness and national sport governing bodies: a
constituency approach [dissertation]. Université d’Ottawa.
Notes
1
Performance is considered here as a more general concept, encompassing both effectiveness and efficiency; it constitutes also an agreed ideal goal.
2
The increasing rationalization of NSOs and the common tendency towards the adoption of a management approach to voluntary organization has reinforced the importance of
efficiency.
3
Studies of strategy, structure, and management methods carried out on Canadian
Organizations by Slack, Thibault, and associates are not included, since they are more focused on the explanation of performance than on the measure of performance. For a synthesis
of this research, see Bayle (3).
4
This perspective is different from Madella (10), whose project was aimed at developing a tool for comparing the performance of a single organization across the years, rather than
different sport organizations at a certain time.
5
For a methodological approach to this subject, see Revat et Madignier (14).
About the Authors
Emmanuel Bayle has a Doctorate in Management and is Director of the Master of
Sports Management Programme at the University of Lyon I. He works on management and
on sports organization performance assessment as well as on the professionalization of sports
and sports organizations.
Alberto Madella has a doctoral degree in sociology. He is General Secretary of the
European Association for Sport Management and works as a consultant for the National
Olympic Committee of Italy and various Italian national sport federations.