ZFVP XX (2008) XX:1–23
DOI 10.1007/s12286-008-0003-4
A ufs ä tze
The Civil Liberty Dataset: Conceptualization and
Measurement
Svend-Erik Skaaning
Zusammenfassung: Dieser Artikel präsentiert ein neues Datenset zu Messung von bürgerlicher Freiheit und diskutiert dieses in methodologischer Hinsicht bezüglich Reliabilität, Validität und Dimensionalität. Der Datensatz umfasst jährliche Angaben von Ende
der 1970er bis 2003 für 28 (post-)kommunistische und 20 lateinamerikanische Länder. Theoretische Grundlage bildet die liberale Theorie. Zur Messung von bürgerlicher
Freiheit werden fünf Indikatoren bestimmt: (1) Unabhängigkeit der Gerichte; (2) freie
Meinungsäußerung; (3) Versammlungs- und Organisationsfreiheit; (4) Gedanken-, Gewissens- und Religionsfreiheit; (5) Bewegungs- und Niederlassungsfreiheit. Mittels statistischer Tests wird gezeigt, dass die Daten eine hohe Intercodiererreliabilität besitzen
und eine gemeinsame latente Dimension aufweisen. Die Studie stellt nicht nur ein neues
Messverfahren und einen neuen Index für bürgerliche Freiheit vor, sie bietet auch einen
genauen Leitfaden für die Erstellung subjektiver Messungen, der Fragen des Fokus, des
Umfangs, der Konzeptualisierung, der Messung und der Aggregation abdeckt.
Schlüsselwörter: Bürgerliche Freiheit · Messung · Datensatz
Abstract: This article presents the construction of a new dataset on respect for civil
liberty and provides in-depth discussions of the methodological issues related to the
reliability, validity, and dimensionality of the scores. The dataset covers 28 (post-) communist and 20 Latin American countries on an annual basis from the end of the 1970s
till 2003. It is theoretically well-grounded in liberal theory and consists of five indicators: 1) independence of courts; 2) freedom of opinion and expression; 3) freedom
of assembly and association; 4) freedom of thought, conscience, and religion; and 5)
freedom of movement and residence. Statistical tests show that the data is characterized by a fairly high degree of inter-coder reliability and that the indicators reflect a
common latent dimension. Apart from proposing a new dataset and an index on civil
liberty, this study offers a meticulous guideline for the creation of subjective measures
that addresses the choices concerning focus, scope, conceptualization, measurement,
and aggregation.
Svend-Erik Skaaning PhD (*)
Department of Political Science, University of Aarhus, Bartholins Allé, bygning 1331,
DK-8000 Århus C, Denmark
E-Mail: [email protected]
S.-E. Skaaning
Keywords: Civil Liberty · Measurement · Dataset
Introduction
The concept of civil liberty, which pervades internationally defined human rights standards, calls for comparison and measurement. Political theoretical/philosophical reflections as well as UN treatises and other international conventions, however, do not
themselves identify each right in sufficient detail to provide a basis for cross-national
measurement and comparison; despite their great significance as a foundation for worldwide agreement on the importance of specific rights. To collect and analyze quantitative
information on a particular human right thus requires a lengthy process of developing
and testing suitable measurement instruments and data processing procedures (Claude
and Jabine 1992: 12). Yet, in the words of Gerardo Munck and Jay Verkuilen (2002:
6, 31):
To a large extent, problems of causal inference have overshadowed the equally
important problems of conceptualization and measurement … [but] the careful
development of measures constitutes the foundation for efforts at drawing causal
inferences and is a critical task in itself.
Because the measurement of core concepts is crucial for the description and explanation
of political phenomena, likely effects of inadequate measurement are illusionary insights instead of the coveted objective: cumulative knowledge gain. The easiest way to
handle the measurement task would be to follow the standard practice, which is to draw
upon readily available datasets. However, the existing datasets and affiliated measures
on the respect for civil liberties are flawed by significant problems (Skaaning 2006a).
Illustrating this point, the disaggregated scores of the widely used Civil Liberty Rating generated by Freedom House are not publicly available and the underlying coding
scheme is merely a checklist that has undergone several changes. Furthermore, Freedom
House neither carries out dimensionality tests nor interrater reliability tests, and its
measure builds upon a very maximalist definition of civil liberty including questionable
components such as personal social freedoms, absence of economic exploitation, and the
right to own property and establish private business (cf. Freedom House 2004). By most
methodological standards, the best alternative to the Freedom House scores has hitherto
been the impressive CIRI Human Rights Dataset, but this dataset is characterized by
a number of shortcomings as well. For instance, the measurement levels of the items
are not justified and rather rough. Freedom of religion and freedom of movement, two
fundamental civil liberties known to face very different degrees of restrictions across
time and space, are even coded in a dichotomous way (presence/absence). To specify
this problem, the lacking ability to distinguish more adequately between different levels
of government repression is one of the main reasons why Germany was assigned the
The author would like to thank Jørgen Elklit, Hans-Joachim Lauth, Gerardo Munck, Jørgen
Møller, Kim Sønderskov, and the anonymous reviewers for their valuable comments and suggestions.
The Civil Liberty Dataset: Conceptualization and Measurement
same score on the Empowerment Rights Index as Georgia, Haiti, and Kyrgyzstan in
2004. Besides, the aggregation rules behind this indexation are hardly supported by any
argumentation (cf. Cingranelli and Richards 2004).
A blatant research challenge thus emerges. The rather insufficient state of the art has
motivated the creation of new data gathered in the Civil Liberty Dataset (CLD). Any
such effort to create a new indicator of civil liberty should be guided by the goals of maximizing validity and minimizing systematic and random measurement errors (Bollen
1993: 1224). An additional criterion is to present the procedures of the data generation
to facilitate quality assessment and ongoing learning processes (King et al. 1994: 8). Yet
too few scholars fulfil these criteria and explain and justify the decisions shaping the
construction of datasets and measures. The generation of data is a very complex undertaking indeed, which involves an iterative process of theorizing and testing, alongside
a wide range of consequential choices (Munck and Verkuilen 2003: 2-3). Consequently,
to make decisions more open to scrutiny, this article offers detailed descriptions and
discussions of the methodological choices connected to the construction of the Civil
Liberty Dataset.
In short, this article presents the construction of an original data compilation covering
Latin American and (post-) communist countries and proposes the associated Civil Liberty Index (CLI). The methodological considerations naturally related to the task, such
as examinations and discussions of reliability, validity, and dimensionality, follow along
this presentation. The article is organized in the four orders of decisions coped with in
the process of data and index construction, that is, the determination of the dataset’s
focus and scope, conceptualization, measurement, and aggregation procedures.
Focus and Scope
The initial task in constructing a dataset is to specify the principal focus of the assessment. To determine the core meaning of civil liberty (cf. Skaaning 2006b), a useful starting point is liberal political thinking in general and Isaiah Berlin’s concept of negative
liberty in particular. He used this concept to settle the area within which a person or a
group is or should be left to do what he or she is able to do or be, without other people interfering (1997: 200-201). David Miller has nuanced Berlin’s distinction between
negative and positive liberty by suggesting an overall tripartition of the main traditions
of views on liberty into republican (self-government), idealist (self-realization), and
The values of the Empowerment Rights Index reported in the CIRI dataset are based on a simple addition of the scores for five items (political participation, freedom of speech, freedom
of religion, freedom of movement, and workers’ rights).
Latin American countries: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, the
Dominican Republic, Ecuador, El Salvador, Guatemala, Haiti, Honduras, Mexico, Nicaragua,
Panama, Paraguay, Peru, Uruguay, and Venezuela. Post-communist countries: Albania, Armenia, Azerbaijan, Belarus, Bosnia-Herzegovina, Bulgaria, Croatia, Czech Republic, Estonia,
Georgia, Hungary, Kazakhstan, Kyrgyzstan, Latvia, Lithuania, Macedonia, Moldova, Mongolia, Poland, Romania, Russia, Serbia-Montenegro, Slovakia, Slovenia, Tajikistan, Turkmenistan, Ukraine, and Uzbekistan (Czechoslovakia, the Soviet Union, and Yugoslavia).
S.-E. Skaaning
liberal (self-protection) freedom. Of these, liberal freedom understands freedom as a
property of individuals and it consists of the absence of constraint or interference by
public authorities and/or other persons in general (1991: 2).
The latter part of this definition hints to the next step in the focus clarification, namely that any given civil liberty measure can be designed from two perspectives. One
concerns government compliance – as a means of measuring whether a government
respects a particular right – and the other concerns individual enjoyment, that is, as a
means of measuring whether each person fully enjoys the right. Although compliance
and enjoyment are two sides of the same coin, it makes sense to approach indicators
for human rights from both angles (Green 2001: 1085-1086) because the enjoyment of
rights can be infringed upon by other parties than the government.
An overview of previous conceptualizations of civil liberty demonstrates that the
constitutive attributes of the basic concept show much convergence. However, flagrant
divergences also exist (Skaaning 2006a). One of the primary differences mirrors a widely acknowledged, although disputed, theoretical (and empirical) division between
personal integrity rights and what I call personal exertion rights, such as freedom of
speech, assembly, and association. The group of personal exertion rights has elsewhere
been labelled basic freedoms (Saward 1994: 16) or – sometimes with a slightly different
meaning – political and/or civil rights/liberties (Shue 1980: 19; Milner et al. 1999: 405;
Landman 2004: 927; Diamond and Morlino 2005: xxv). The Civil Liberty Dataset is
meant to cover the latter type of rights, that is, the extent to which the state restricts certain freedoms understood as actions that individuals or groups might wish to perform.
Thus, civil liberty is henceforward primarily understood as something different from,
although associated with, personal integrity rights and other alternative conceptualizations of civil liberty/civil rights (cf. Waldron 2003: 195). After the introduction of this
restriction, the core concept in question is seen to encompass only one principal dimension – in contrast to most definitions of political democracy (e.g., Dahl 1971: 4-8).
When constructing the CLD, neither the general conditions of the respective civil liberties nor the existing legal protections affect the coding. In other words, the
population’s inability to utilize the liberties due to lack of initiative, commitment, financial means, or the like does not influence the score. Similarly, the presence or absence of
formal guarantees of civil liberties in the constitution or formal legislation is only taken
into consideration when observed by the public authorities. Only the actual practices of
governments/states (public authorities in the broadest sense) and their agents, meaning
their concrete performance regarding respect for and guarantee of civil liberty, are assessed. Furthermore, the geographic distribution of civil liberty violations in a country
does not affect the assessment but, on the other hand, only violations that occur within
the internationally recognized borders of a country are considered. In addition, it is
considered a civil liberty violation if large (mainly ethnically defined) groups without
citizenship in the country where they have resided for many years do not hold the same
rights as the citizens of the particular country.
This distinction is also reflected in the disposition of international and regional human rights
conventions.
The Civil Liberty Dataset: Conceptualization and Measurement
As for the scope of the dataset, the immature state of knowledge about the development in and explanation of civil liberty makes it vital to facilitate both intra- and interregional comparisons. Therefore, the obvious choice is to focus on countries from two
distinctive and broad regions that have been swept by the recent and significant regime
changes, namely Latin America and post-communist Eastern Europe. In this way, the
dataset can contribute to the debate on the status of the ‘transition paradigm’ as well as
the fruitfulness of intra- and interregional comparisons as represented by the positions
of Phillippe Schmitter and Terry Karl (1994; Karl and Schmitter 1995) on the one side
and Valerie Bunce (1995a; 1995b) on the other.
Inspired by their dispute, the dataset (tentatively) covers 20 Latin American and 28
post-communist countries and the construction is based on the premise that it is important to study the intra- and interregional differences and similarities in the levels of and
developments in civil liberty. Furthermore, the CLD is prioritized to cover the regime
changes linked to the third wave of democratization in Latin America as from the mid1970s and the widespread regime changes in the communist countries behind the Iron
Curtain initiated in 1989. Even though it is inconvenient, the completion of a research
process demands that a (tentative) final year is selected, which in this case is the year
the project was launched, namely 2003.
Conceptualization
The task of conceptualization – specifying the meaning of the core concept by identifying constitutive attributes – affects the entire process of data generation, given that
it provides the anchor for all subsequent decisions (Munck and Verkuilen 2002: 7).
There are apparently two main ways to approach a definition of civil liberties: either to
derive a descriptive definition from the characteristic features of existing regimes or to
base a conceptual definition on the history and philosophy of liberal thought. The latter
approach is preferred because it rests on abstract values rather than depending on value
judgments about present regimes (cf. Forewaker and Krznaric 2001: 3). Then again, the
outcomes from either approach are not expected to diverge much.
A detailed exploration into liberal political theory/philosophy, general global and regional human rights conventions, and key historical events in the agenda setting of civil
liberties (Skaaning 2006b) identified five key civil liberties that the government/state
is not allowed to restrict its citizens from performing. Consequently, these aspects are
selected to constitute the items of the CLD. The assessed rights expected to reflect a
political regime’s liberality are: 1) independence of courts; 2) freedom of opinion and
expression; 3) freedom of assembly and association; 4) freedom of thought, conscience
and religion; 5) freedom of movement and residence (see figure 1).
These fundamental civil liberties have, more or less without exception, been emphasized by liberal theorists from the Enlightenment and onwards as well as by the
authors of the predominant international human rights conventions. For instance, Benjamin Constant associated (civil) liberty with the right to be subjected only to laws,
to express one’s opinion, to come and go without permission, to associate with other
individuals, and to practice one’s religion of choice (1988: 311). John Rawls’s battery of
S.-E. Skaaning
Civil Liberty
Independence
of courts
The extent to which citizens and groups have the right to be
under the jurisdictionof, and to seek redress in, courts that can
review and interpret the law and pass judgments without being
subject to actual limitations or restrictions
Freedom of
opinion and
expression
The extent to which citizens, groups, and members of the press
have the right to hold views freely and to seek, obtain, and pass
on information on political issues broadly understood without
being subject to actual limitations or restrictions
Freedom of
assembly and
association
The extent to which citizens have the right to gather freely and
carry out peaceful demonstrations as well as to join, form, and
participate with other persons in political parties, cultural
organizations, trade unions, or the like of their choice without
being subject to actual limitations or restrictions
Freedom of
thought,
conscience, and
religion
The extent to which citizens have the right to have and change
religion or belief of their own choice and alone or in a community
manifest their religion or belief in practice, worship, observance,
and teaching in private or public as well as proselytizepeacefully
without being subject to actual limitations or restrictions
Freedom of
movement and
residence
The extent to which citizens have the right to settle and travel
within their country as well as to leave and return to their
country of own choice without being subject to actual
limitations or restrictions
Figure 1 Component Items and their Definitions
basic civil liberties includes moral liberty and freedom of thought, belief, and religious
practice; freedom of speech and assembly; liberty to form associations; and the rule of
law understood as regular and impartial administration of law (1971: 211-212, 222-224,
235-243), while the liberties shown in figure 1 are virtually identical to the list of basic
freedoms accentuated by Michael Saward (1994: 16). Both the abstract meaning and
the specific identification of civil liberties thus rest on a solid foundation building on
liberal political theory.
Any definition of civil liberty should address its internal relationship to the concepts
of human rights and democracy. As it is beyond the scope of this article to go into detail
with this issue, a condensed illustration of my point is shown in figure 2. On the highest
level of abstraction we find six human rights categories, written in upper-case letters.
Elsewhere they have been divided into three generations of human rights according to
their historical sequence (Umozurika 1998: 539).
The first refers to civil liberty and democracy; the second to economic rights, social rights,
and cultural rights; and the third to solidarity rights such as peace and environmental protection. The placement of the latter four types of human rights in the different corners does not
indicate theoretical or empirical ‘proximity’ to the other concepts.
The Civil Liberty Dataset: Conceptualization and Measurement
CULTURAL RIGHTS
ECONOMIC RIGHTS
dignity
freedom of movement
inclusive
participation
civilian supremacy
security
freedom of
religion
competition
national sovereignty
independent courts
CIVIL LIBERTIES
Personal Integrity Rights
freedom of
assembly and
association
freedom of
expression
Personal Exertion Rights
Political
Liberties
SOLIDARITY RIGHTS
DEMOCRACY
Access to
Political Power
Accountable
Government
SOCIAL RIGHTS
Figure 2 Civil Liberties and Democracy in the Human Rights Landscape
On the next level, written in italics, personal integrity rights and personal exertion
rights are subsets of civil liberties, while – inspired by Bowman et al. (2005) – access
to political power, accountability of government, and political liberties are emphasized
as different features of democracy. The concrete rights, written in lower-case letters,
make up the lowest level of abstraction. We see that freedom of expression and freedom
of association and assembly are both considered to be civil liberties (personal exertion
rights) and democratic rights (political liberties), while freedom of movement and freedom of religion are only civil liberties in the form of personal exertion rights. Finally,
judicial independence, integrity, and impartiality, here called the right to independent
courts, is a civil liberty linked to personal integrity as well as personal exertion.
The relation between civil liberties and democracy can be summarized in different
ways (Bobbio 1990: 48-49). Some see civil liberties and democracy as distinct phenomena; that is, democracy is only associated with political self-governance (access to
political power and sometimes accountability of government), which can exist, but does
not necessitate, simultaneous respect for civil liberties. Berlin (1997: 201-203) argues
that liberty in the negative sense is compatible with some kinds of autocracy or, at any
rate, with the absence of self-government because civil liberty concerns the area of
control, whereas self-government concerns its source. In short, there is a logical distinction between answering the questions about who governs and the extent of government
interference in the life of the citizens, respectively (Böckenförde 1991: 365).
Yet, Jürgen Habermas (1992: 610-616; 1996: 294-301), among others, has criticized
this schism between civil liberties and popular sovereignty as being flawed because certain rights are constitutive conditions for free political opinion formation and
decision-making and thus immanent prerequisites for the meaningful functioning of
S.-E. Skaaning
self-government. In line with this argument, Robert A. Dahl (1971: 3) incorporates
freedom of speech and press as well as freedom of association and assembly in his polyarchy criteria, while – inspired by Habermas – Hans-Joachim Lauth (2004) includes
the rule of law (independent courts) as well. Some even consider most, if not all, human
rights as democratic rights (Beetham et al. 2001). No matter which relation is judged to
be more reasonable, however, it is plausible to treat the features as analytically distinct
in theoretical and empirical studies. In addition, all conceptulalizations involve tradeoffs, meaning that a choice between maximalist and minimalist strategies – linked to
different advantages and weaknesses (Munck and Verkuilen 2002: 9) – must first and
foremost depend on the research question at hand.
Measurement
Even though the conceptual challenge has been tackled, it is far from evident how to
assess whether countries can be characterized by the state’s observance of civil liberties.
Before the actual assignment of scores, a number of decisions concerning measurement
level, coding guidelines, and sources must be taken in order to facilitate the coding
process and to improve the consistency and utility of the dataset.
As regards the first issue, the measurement level, all items are coded on the basis of
a four-point scale with two endpoints. The four points theoretically refer to an overall
distinction between ideal typical characteristics of liberal, semi-liberal, illiberal, and
anti-liberal regimes (see figure 3). This design is more or less parallel to the division of
political regimes into democratic, semi-democratic, authoritarian, and totalitarian types
(cf. Merkel and Crossaint 2000; Lauth 2004: 321). Whereas the classical tripartition of
political regimes into democratic, authoritarian, and totalitarian regimes used to play
– and, indeed, still plays – a dominant role in many classification attempts (Linz 2000;
Neumann 1957: ch. 9; Merkel 1999: ch. 1), a slight revision has lately been suggested
by numerous scholars.
Based on research on contemporary regimes, the need for an intermediate category
between democracies and authoritarian regimes has been emphasized because many
present-day countries are only inadequately covered by any of these terms (Schedler
2002; Diamond 2002; Case 1996; Karl 1995; Zakaria 1997; Ottaway 2003; Levitsky
and Way 2002; Merkel et al. 2003). There is no reason to believe that this development
is not present when it comes to civil liberty since many of the adjectives attached to the
democracy concept, such as illiberal, hard, and electoral, refer to inadequate protection
of one or more civil liberties (cf. Collier and Levitsky 1997: 438-441). Thus, it seems
reasonable to include the semi-liberal category and, accordingly, introduce a four-point
1
2
3
4
Anti-liberal
(severely restricted)
Illiberal
(fairly restricted)
Semi-liberal
(modestly restricted)
Liberal
(unrestricted)
Figure 3 Measurement Scale
The Civil Liberty Dataset: Conceptualization and Measurement
assessment scale. The term anti-liberal, which corresponds to the widely disputed term
totalitarian, is used for two main reasons. First, it seems to make up a more proper
endpoint of the scale, and second, some of the non-democratic regimes in Latin America and many, if not all, of the communist regimes and a few of the post-communist
regimes have been deemed totalitarian instead of authoritarian by many scholars.
A more detailed fixed set of coding standards was elaborated that makes the respective scales correspond more straightforward to concrete circumstances. The outcome
is outlined in a coding manual inspired by the comprehensive codebook offered by
Cingranelli and Richards (2004) in connection with their Human Rights Data Set. At
first, several exploratory uses and discussions of this codebook’s guidelines in teaching
sessions uncovered some of its advantages and shortcomings. This procedure then led
to the drafting of a codebook used to guide the coding process (see extract in the appendix). The codebook was finally subjected to minor adjustments in an iterated fitting
process (cf. Bowman et al. 2005: 957), in which the description of the individual scale
points for each item was refined to fit specific contexts and the way information was
presented in the sources.
The availability of such annual sources on civil liberty observance in the countries
of interest is rather poor and the existing sources diverge very much as to the amount
and quality of the information provided. However, the U.S. State Department’s annual
publication, the Country Reports on Human Rights Practices, is rather consistent across
nations, over time, and vis-à-vis the different aspects covered, including the issues covered by the component items. The reports are based on information available from a
wide variety of sources, among them U.S. and foreign government officials, victims of
human rights abuse, academic and congressional studies, and reports from the press,
international organizations, and nongovernmental organizations concerned with human
rights. In an initial phase, U.S. embassies gather such information throughout the year
and prepare the initial drafts of the reports. The reports are then reviewed by the Bureau
of Democracy, Human Rights, and Labor, in cooperation with other State Department
offices, which draw on their own sources of information such as reports provided by
U.S. and other human rights groups, foreign government officials, representatives of the
United Nations and other international and regional organizations and institutions, and
experts from academia and the media.
The final reports assess the degree to which human rights standards are respected in
countries around the world and have, unsurprisingly, been a continuing source of controversy since they were first issued in the mid-1970s. The first two reports, those issued
in 1975 and 1976, were severely criticized by Congress, which officially turned to the
Library of Congress for alternate information. Consequently, substantial changes were
made in the preparation of the reports, which significantly improved the scope, quality,
and independence of the subsequent reports (McNitt 1988: 97-98). In any case, critics
such as the Lawyers Committee for International Human Rights have frequently charged the State Department with biased reporting. The prominent accusation has been that
the level of repression in countries ideologically opposed to the United States is overestimated, whereas countries in which the U.S. has compelling interests are favored.
Yet careful and critical examinations tend to agree that the annual State Department
reports are an invaluable source accurately reporting on the conditions in most of the
10
S.-E. Skaaning
countries most of the time (cf. Poe et al. 2001: 650-651). A qualitative comparison
with two other prominent sources of comprehensive cross-national information on a
broad spectrum of human rights practices, namely the annual reports by Amnesty International and Human Rights Watch, shows that the Country Reports on Human Rights
Practices are by far the most detailed and complete. They are not only more extensive
and systematic than those of the other monitoring organizations, but they also cover
more countries and human rights issues. Actually, the alternatives hardly ever provide
usable information on the civil liberties examined in this study, so their inclusion as
extra sources would yield little.
Furthermore, a quantitative evaluation of the difference between scores obtained by
coding the annual reports provided by the State Department and Amnesty International
with regard to personal integrity rights showed that the Country Reports on Human
Rights Practices have at times been politically biased but, that being said, there is no
reason to believe that State Department biases have affected the description of the vast
majority of cases in any systematic way (Poe et al. 2001: 661, 670). Besides, the bias
that appeared in some of the initial State Department reports in the 1970s and early
1980s has tended to disappear over time (Poe et al. 2001: 677). In keeping with this
finding, Judith Innes (1992) has shown that accurate and unbiased reporting on human
rights issues has become an intrinsically important goal for many key actors within
the State Department. Public debate over the information among experts and human
rights organizations has increased consistency in definition, accuracy in measurement,
and comprehensiveness of coverage over the years. Thus, the reports have come to be
generally independent of the administration’s political stance.
Even though it is preferable to base the dataset on more than one source in order
to increase the amount of information and diminish the influence of potential bias (cf.
Bowman et al. 2005), resource matters (time, etc.) meant that the State Department’s
country reports were selected as the only source. One implication of this choice was
that the dataset does not go further back in time than 1977 for most Latin American
countries and 1979 for all communist countries except Yugoslavia (1977). The possibility for other scholars to scrutinize the coding through replication, though, is rather
high due to the easy access to the main source, of which the relevant parts make up
their own subsections.
In transforming the information into the scores constituting the dataset, at least two
independent coders assigned scores to all the country-years. At the time of the coding
process, one of the coders, the present author, was a Ph.D. scholar in political science
The amount of information included in the reports has significantly increased since the beginning, so they now range from approximately 10 to 100 pages for each country with a tendency to be more comprehensive for large and/or trouble-ridden countries. Also, the number
of issues assessed has increased during the years, but the five topics of interest have always
been covered by the reports.
This is probably the main reason why Cingranelli and Richards (2004) only use the Amnesty
International reports as sources for the coding of personal integrity rights and not of personal
exertion (empowerment) rights included in their CIRI Human Rights Dataset.
The first year of assessment for Argentina, Brazil, Chile, and Cuba is 1979.
The Civil Liberty Dataset: Conceptualization and Measurement
11
specialized in comparative methodology and regime development in Latin American
and post-communist countries. I assessed all the country-years in question, whereas the
second assignment was split equally between two trained master’s students (political
science) with a broad knowledge of methodological issues and the countries considered.
Deviations were settled through subsequent discussions. In the event that this procedure did not result in agreement, the plan was to authorize the third coder with the final
judgment. This procedure, however, was not applied because consensus was eventually
reached with regard to all country-years and items, that is, 4,930 scores.
Inter-coder reliability, meaning the extent to which independent coders agree in their
evaluation, is a necessary although not sufficient criterion for validity, and without it,
the uncertainty of the results obtained using the dataset increases tremendously. The
most simple and widespread measure of inter-coder reliability is to calculate the percentage of agreement. However, in the methodological literature it is seen as misleading and inappropriately liberal, so a measure known to be rather conservative, namely
Cohen’s Kappa, is also employed. As the principal source for some fairly similar CIRI
indicators is the same (the Country Report on Human Rights Practices), the correlation
(Gamma) between these and the civil liberty indicators is another means of assessing
the reliability. The results of these tests, separate for each variable and region, are
shown in table 1.
The values indicate that the inter-coder reliability of the Civil Liberty Dataset is
acceptable. The Cohen’s Kappa test coefficients almost all exceed 0.80, while the coefficients for the Latin American data are all lower than their post-communist counterparts.
A likely explanation for this difference is that the scores referring to Latin American
countries to a larger degree reflect the coding of less informative older reports, while it
has simply been easier to evaluate the almost unexceptionally harsh repression level of
Table 1: Inter-Rater Reliability Statistics
Latin American
Independence of Courts
Freedom of Opinion and
Expression
Freedom of Assembly
and Association
Freedom of Thought,
Conscience, and
Religion
Freedom of Movement
and Residence
Cohen’s
Kappa
Percentage
Agreement
0.803
0.738
Post-Communist
Cohen’s
Kappa
Percentage
Agreement
0.876
0.855
Correlation
with CIRI
Indicator
0.807
0.891
0.845
0.823
0.892
0.877
Correlation
with CIRI
Indicator
0.860
0.924
0.773
0.850
0.933
0.861
0.903
0.949
0.811
0.914
0.921
0.817
0.881
0.894
0.807
0.887
0.925
0.856
0.901
0.906
The indicators are: Independent Courts, Freedom of Speech and Press, Freedom of Assembly
and Association, Freedom of Religion, and Freedom of Movement.
12
S.-E. Skaaning
communist regimes.10 Additionally, in the Latin American context it has frequently been
uncertain whether violations have been committed by government agents or others and
whether the actions taken against citizens have been based on legitimate reasons. The
values linked to two items, freedom of opinion and expression and freedom of assembly
and association, even fall below the suggested benchmark. Yet, as the levels are still
rather high and considering the well-known conservativeness of the measure, the interrater reliability is found to be appropriate. This judgment is also supported by the high
correlations with the CIRI indicators and the fact that none of the differences among
coding scores was more than one point. The somewhat lower correlation between the
indicators reflecting the independence of courts is probably due to a higher degree of
discrepancy in the respective definitions.
Aggregation
The disaggregated data are, together with the codebook, available on the project website (www.democracy-assessment.dk), allowing researchers and others interested in civil
liberty issues to scrutinize the assessments and to use and them as they please. Instead
of concluding the presentation of the new dataset now, I will continue with the construction of an index. The reason is that, despite the loss of information, we often use
composite measures to climb up the ladder of abstraction and/or to minimize problems
with multicollinearity, degrees of freedom, etc. Before combining the indicators and
constructing an overall index, however, the dimensionality of the dataset is examined
in order to uncover whether the indicators not only conceptually but also empirically
reflect a common latent dimension, so an aggregation does not force a multidimensional phenomenon into one composite measure. The coefficients in table 2 show the
correlation (Gamma) between the component items, where the coefficients related to
Latin American and post-communist countries are found above and below the diagonal,
respectively.
With regard to the post-communist data, all coefficients exceed 0.84, indicating strong
unidimensionality. Actually, the level of some of the correlations, for example between
the indicators for the independence of courts and freedom of expression, are so high that
their scores are virtually equivalent. The results for the Latin American data are more
mixed. The coefficients differ more – ranging from 0.467 to 0.871 – and are generally
lower, but then again, on a relatively high level. The covariation between independent
courts and freedom of movement to some extent constitutes an exception.
The items appear to tap into the same dimension and to support this claim further.
Examinations of the dataset were carried out by calculating the item-total11 correlations
(Gamma) and Cronbach’s alpha in addition to running a factor analysis (principal component) (see table 3). With regard to the post-communist countries, the items tend to
be extremely unidimensional. The scope of the item-total correlations is 0.825-0.955;
10 The latter reason finds more support in year by year calculations of the inter-rater reliability
(Kappa) than the former.
11 A tentative index constructed through simple addition of the items except the item in question.
The Civil Liberty Dataset: Conceptualization and Measurement
13
Table 2: Dimensionality Statistics – Item-Item Correlations
Independence
of Courts
Independence of
Courts
Freedom of
Opinion and
Expression
Freedom of
Assembly and
Association
Freedom of
Thought,
Conscience, and
Religion
Freedom of
Movement and
Residence
Freedom of
Movement
and Residence
0.656
Freedom of
Thought,
Conscience,
and Religion
0.584
0.871
0.659
0.722
0.553
0.741
Freedom of
Opinion and
Expression
Freedom of
Assembly and
Association
0.802
0.974
0.927
0.939
0.884
0.916
0.910
0.847
0.904
0.851
0.467
0.610
0.848
Cronbach’s alpha value is 0.937 and the removal of any item would reduce it. Furthermore, the factor analysis only extracts one dimension12 and the factor loadings of the
items are very high.
Analyses of the Latin American data also support that the items reflect the same
underlying phenomenon in that the factor analysis only extracts one component, and
Cronbach’s alpha is as high as 0.859. The overall picture for the Latin American data,
though, is a bit more blurred as illustrated by the generally lower item-total correlations and factor loadings. Two items, namely independence of courts and freedom of
thought, conscience, and religion, generally appear less associated with the latent dimension. Nevertheless, they are used in the index construction because the items ensure
analytically crucial and interesting variation (especially in the Latin American context)
and because the conceptual analysis underscored their coherence with the other items,
which underlined the implausibility of any exclusion. With an alpha value of 0.915 and
high correlations and factor loadings, similar tests including all countries confirm the
region-specific findings.
In order to explore whether the high correlations between the items are artificially
boosted by a high degree of stability across the years and countries in question, as was
12 Post-communist: Principal component. The eigenvalue of the first component is 4.011, explaining 80.2% of the variation, whereas the eigenvalue of the second component is only
0.343, explaining less than 7% of the variation. Latin American: Principal component. The
eigenvalue of the first component is 3.232, explaining 64.6% of the variation, whereas the
eigenvalue of the second component is only 0.666, not even explaining 14% of the variation.
All: Principal component. The eigenvalue of the first component is 3.745, explaining 75% of
the variation, whereas the eigenvalue of the second component is 0.487, explaining less than
10% of the variation.
14
S.-E. Skaaning
Table 3: Dimensionality Statistics Continued
Independence of
Courts
Item-Total
0.584
Cronbach’s Alpha
0.858
Factor Loadings
0.705
Item-Total
0.916
PostCronbach’s Alpha
0.921
Communist
Factor Loadings
0.896
Item-Total
0.756
All
Cronbach’s Alpha
0.911
Factor Loadings
0.806
Note: Cronbach’s Alpha values if item deleted.
Latin
American
Freedom of
Opinion and
Expression
Freedom of
Assembly and
Association
0.824
0.804
0.875
0.955
0.912
0.930
0.913
0.880
0.918
0.737
0.808
0.863
0.904
0.918
0.912
0.819
0.887
0.896
Freedom of
Thought,
Conscience, and
Religion
0.561
0.845
0.752
0.875
0.928
0.875
0.753
0.901
0.844
Freedom
of Movement and
Residence
0.617
0.827
0.812
0.825
0.930
0.863
0.761
0.897
0.859
seemingly the case under communism, the factor loadings (principal component) for
each year since 1979 have been calculated. The results for 2003 and every fifth year
back to 1983 shown in table 4, however, indicate that this is not a critical issue.
The dimensionality statistics also show that freedom of opinion and expression and,
modestly less, freedom of assembly and association seem to be the best individual
predictors of the level of civil liberty. This finding is interesting because, besides being
defining attributes of the civil liberty concept, the freedoms are also constitutive parts of
many democracy definitions (cf. figure 2 above); a high covariation between the respect
for democratic and civil liberties (and, to some extent, similar causes and effects), at
least in the countries and the period under consideration, is therefore to be expected.
Concerning the actual aggregation rule, simple addition (equals arithmetic mean) is
preferred because the component items are considered to have the same weight. Depending on the context, some people find some of the civil liberties in focus more important,
while others would come to a different order of priority. In political theory/philosophy,
Table 4: Factor Loadings in Selected Years
Latin American
Year
Inde- Freedom FreeFreepenof Opi- dom of
dom of
dence nion and Assem- Thought,
of
Expres- bly and ConsciCourts
sion Associa- ence, and
tion
Religion
1983
1988
1993
1998
2003
0.867
0.762
0.713
0.614
0.661
0.942
0.909
0.902
0.803
0.858
0.906
0.913
0.833
0.792
0.900
0.703
0.775
0.653
0.810
0.770
Post-Communist
Freedom of
Movement
and
Residence
0.830
0.815
0.698
0.852
0.773
IndeFreeFreeFreeFreependom of dom of dom of dom of
dence Opinion Assem- Thought, Moveof
and
bly and Consciment
Courts Expres- Associa- ence, and
and
sion
tion
Religion Residence
0.830
0.937
0.681
0.551
0.755
0.975
0.975
0.856
0.637
0.945
0.870
0.929
0.859
0.651
0.813
0.908
0.859
0.866
0.837
0.769
0.915
0.901
0.838
0.887
0.809
The Civil Liberty Dataset: Conceptualization and Measurement
15
the prevailing tradition is also not to weight or prioritize among them. Furthermore, the
liberties are not assumed to be mutually constitutive. This means that if one of them
is given a low value, it does not necessarily affect the interpretation of another item.
In order to obtain full membership in the group of liberal regimes, however, all civil
liberties must be fully observed. Moreover, all the items have end points, thus limiting
the possibility of compensation/substitution in significant and reasonable ways.
The robustness of the aggregation rule is examined by comparing (correlating) the
Civil Liberty Index (CLI) with six different ways of combining the five items. Among
the alternatives are the geometric mean, which is relevant when several quantities multiply together to produce a product, and the factor scores (principal components) linked to the dimensionality assessment. Yet another procedure, multiplication of the item
values, is suitable if each component item must have a certain minimum to make the
other affiliated aspects meaningful.13 A related manner of reasoning applies to the use
of the minimum value as index score. In this case, the logic of necessity is used on
all four item levels (scale points), whereas – in contrast – use of the maximum value
corresponds to identifying all items as individually sufficient. The final option is a
weighted simple average, where the scores for freedom of opinion and expression and
freedom of assembly and association are given double weight because they may be judged more important due to their twofold function as both civil and democratic liberties.
The correlation coefficients (Pearson) are shown in table 5.
The results indicate that it can make a considerable difference which aggregation
rule is chosen. The three alternatives connected to necessity or sufficiency argumentation lead to coefficients below 0.9 but not lower than 0.7. As it is rather obvious that
none of the individual items are sufficient for civil liberty, the aggregation procedure of
equalizing the index score with the maximum score is an implausible route to follow. If
the main concept was constituted by two or more mutually constitutive dimensions, it
would (more likely) be proper – indeed, maybe even indispensable – to regard different
elements as necessary and thus try to adjust the composite measure by using some sort
of multiplication or minimum procedure in the aggregation process. According to Gary
Goertz (2006), the minimum procedure is probably the proper way to operationalize
our concepts in most cases.
However, although his arguments are certainly analytically appealing, I do not regard
the civil liberty aspects to be necessary in the sense of being fully mutually constitutive.
In my view, Goertz’s point is more relevant in relation to participation and competition
Table 5: Correlations between CLI and Indices Based on other Aggregation Methods
or Data
Latin American
Post-Communist
All
Geometric
Mean
0.995
0.999
0.997
Multiplication
Factor
Scores
Minimum
Maximum
Weighted
CIRI
FH
0.727
0.750
0.734
0.999
1.000
1.000
0.760
0.912
0.835
0.807
0.885
0.874
0.992
0.997
0.995
0.841
0.838
0.856
-0.835
-0.924
-0.887
13 The items were rescaled (0-3) in order to support this logic.
16
S.-E. Skaaning
in the case of democracy, for example (cf. Bowman et al. 2005). Whereas it does not
make much sense in terms of democraticness to have inclusive participation without
contestation, as was to some extent the case under communist regimes, citizens’ freedom of movement still makes sense without freedom of expression. Furthermore, when
measuring the overall respect for civil liberty, we would lose more information through
the application of the minimum procedure as only four scores would be viable. It also
seems rather implausible to equalize the level of civil liberty in a country assigned only
one low score (e.g., 1, 4, 4, 4, 4) to the level in a country assigned low scores on all
items (e.g., 1, 1, 1, 1, 1), and even concerning the measurement of democracy, application of the minimum rule has forcefully been questioned (Marsteintredet 2007). Finally,
if the main concept only covers one dimension, as theoretically and empirically justified
with regard to civil liberty, it makes less sense to apply alternative procedures. As it
makes virtually no difference if the arithmetic mean is replaced by the geometric mean,
factor scores, or the weighted index, and because it is easy to interpret and handle, I
thus prefer simple addition as the principal rule of aggregation.
Claims for the validity of measures are commonly based on their high statistical
correlations with other measures of the same phenomenon. Yet highly correlated measures may all contain the same errors, share similar biases, or be determined by outside
influences that may render their close associations spurious (cf. Forewaker and Krznaric
2001: 5; Munck and Verkuilen 2002: 29). The same points apply to uncritical use of correlations between different indicators expected to reflect the concept in question. Simple
correlations are not able to capture all kinds of (theoretical/empirical) relationships and
high correlations can be the result of a tendency of clustering, when cases tend to be
assigned very high and very low scores and while crucial disagreements exist for cases
in the ‘gray zone’ of intermediate scores (Goertz 2006). Basically, strong theoretical
justifications in every step of construction as well as reliable empirical information are
obviously the most essential parts of sound datasets and measures, but in many cases
different factors force us into trade-off situations distancing the results from the ideal.
Thus, researchers should refrain from interpreting such correlations as much more than
indications of reliability – to the extent that these measures build on the same sources
– and as indications of whether the results would be significantly different if the measures were interchanged in statistical models.
It is interesting, though, that the CLI, in spite of the differences, is highly correlated (Pearson) with rival measures such as Freedom House’s (FH) (2006) Civil Liberty
Rating14 and Cingranelli and Richards’s (CIRI) (2006) Empowerment Rights Index15,
also shown in table 5. The strong correlations in the range of 0.83 and 0.9216 indicate
14 A composite index based on assessments of freedom of expression and belief, associational
and organizational rights, rule of law, and personal autonomy and individual rights.
15 A composite index based on assessments of freedom of speech, freedom of religion, freedom
of movement, freedom of political participation, and workers’ rights.
16 The yearly correlations with the Freedom House index is fairly stable in both regions, while
the correlations with the CIRI index are stable across the assessed years for the Latin American countries but comparatively lower in the early 1980s and the early 1990s for the (post-)
communist countries.
The Civil Liberty Dataset: Conceptualization and Measurement
17
that the measures very much tap into the same latent dimension. They are not fully
interchangeable, though, so the application of the CLI in empirical analyses as a dependent and explanatory variable would lead to somewhat different results, just as it has
been shown to be the case with democracy indices (Hadenius and Teorell 2004; Bollen
and Paxton 2000; Casper and Tufis 2003). The notably lower correlation between the
measures concerning Latin American countries is probably caused by differences in
constituent parts of the indices/scales. In other words, these differences have a greater
effect in this setting due to the lower extent of unidimensionality compared with the
post-communist countries, which are very much characterized by a tendency that all
good (and, inversely, bad) features coincide.
Limitations on the agenda and scope of this article means that an inquiry into the
explanatory consequences of the new dataset are addressed in detail elsewhere, where
the CLD and the CLI – as well as some of their alternatives – are employed to track
and explain the development in civil liberties (Skaaning 2006c). Nonetheless, to give
an impression of whether it would make a difference if the minimum scores, the CIRI
Empowerment Rights Index, or the Freedom House Civil Liberties Rating were used
in the place of the CLI, the respective standardized scores (0-100) for 2003 of all the
assessed countries are shown in table 6.
Without going into a detailed discussion of the scores, it is obvious from the table
that the employment of the minimum aggregation rule implies a loss of information and
leads to a quite different ranking and score of the countries than the CLI. The procedure
means that the cases can only achieve the scores 0, 33, 66, and 100 and tend to cluster
in the two middle categories. Although the resulting groups are somewhat reasonable,
the set of countries with the lowest score (i.e., Cuba, Haiti, Belarus, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, and Uzbekistan) tends to be more homogeneous than
the rest.
As for the other measures, they tend to agree more. As we would anticipate, the
Central Asian and Caucasian countries show a poor record, and the new EU members
outperform the others. Also not very surprising, Costa Rica and the Southern Cone
countries (Chile, Argentina, and Uruguay) are among the best, while Cuba and Haiti
have much to improve. Noteworthy differences do exist, though. For example, Freedom House assesses the respect for civil liberty in Haiti much lower than the CLI and
the CIRI measure. This dissimilarity could, however, be the result of Freedom House
focusing on all civil liberty violations and not only the ones committed by government
agents. Therefore, it is more disturbing that in four cases (Chile, Peru, Armenia, and the
Ukraine) the CIRI index score is more than 30 points lower than the corresponding values of the Freedom House measure and the CLI. It is also puzzling that Turkmenistan,
with an exceptionally repressive regime, achieves the same score in the CIRI index as
Russia and even better than Armenia, Azerbaijan, and the Ukraine. Furthermore, the
case of Belarus illustrates the problems connected to a dichotomous coding of freedom
of religion and freedom of movement, since it is assigned the lowest possible combined
score even though these civil liberties are not severely repressed.
To illustrate, in brief, that the CLI not only outperforms the alternatives on the descriptive level but also provides higher criterion validity, I examine the relationship
between the measures and an operationalization of the most persuasive theoretical
18
S.-E. Skaaning
Table 6: Comparison of Index Scores (2003)
CLI
MiniCIRI
mum
Argentina
80
33
100
Bolivia
73
33
70
Brazil
73
33
90
Chile
93
67
60
Colombia
60
33
50
Costa Rica
93
67
80
Cuba
7
0
0
Dom. Rep.
80
67
60
Ecuador
67
33
80
El Salvador
73
33
80
Guatemala
67
33
50
Haiti
53
0
50
Honduras
67
33
70
Mexico
73
33
50
Nicaragua
80
33
80
Panama
80
33
70
Paraguay
67
33
80
Peru
60
33
30
Uruguay
93
67
80
Venezuela
60
33
60
Albania
60
33
80
Armenia
60
33
10
Azerbaijan
47
33
10
Belarus
20
0
0
Note: All index scores are rescaled to a
liberty.
FH
83
67
67
100
50
83
0
83
67
67
50
17
67
83
67
83
67
67
100
50
67
50
33
17
range
CLI
MiniCIRI
FH
mum
Bosnia-Her.
53
33
40
50
Bulgaria
73
67
60
83
Croatia
80
67
80
83
Czech Rep.
100
100
80
83
Estonia
93
67
100
83
Georgia
53
33
50
50
Hungary
80
67
90
83
Kazakhstan
40
0
30
33
Kyrgyzstan
40
0
50
33
Latvia
73
67
80
83
Lithuania
80
67
90
83
Macedonia
67
67
80
67
Moldova
53
33
50
67
Mongolia
73
67
60
83
Poland
87
67
70
83
Romania
67
33
50
83
Russia
40
33
20
33
Serbia-Mon.
60
33
80
83
Slovakia
73
67
100
83
Slovenia
87
67
90
100
Tajikistan
33
0
30
33
Turkmenistan
13
0
20
0
Ukraine
53
33
10
50
Uzbekistan
27
0
0
17
of 0-100, 100 designating the highest level of civil
account for civil liberty in the post-communist countries, namely the communist legacy
(Kitschelt 2003; cf. Skaaning 2006c). Following Herbert Kitschelt (2001), I distinguish
between different kinds of legacies, ranging from bureaucratic-authoritarian to highly
patrimonial types, differentiated by the Weberian ‘bureaucraticness’ of the state as well
as the power balance between communists and the (liberal, oppositional) civil society.
These features are traceable to the pre-communist era and were jointly expected to shape the regime trajectories after the communist breakdowns in 1989-1991. Supporting
the construction of a new measure on civil liberty, Kitschelt’s assessment of the vital
(pre-)communist legacies shows a significantly higher correlation with the CLI than the
indices from CIRI and Freedom House – both in the short term (1993: 0.68, 0.60, and
0.61, respectively) and intermediate term (2003: 0.77, 0.69, and 0.70, respectively).
The Civil Liberty Dataset: Conceptualization and Measurement
19
Conclusion
Recapitulating the limitations of alternative measures already addressed in the introduction, these concerns support the construction and use of the Civil Liberty Dataset
and the Civil Liberty Index presented in this article, which demonstrated the decisions
made in order to provide for improved measurement of civil liberty. The prime goal
was to develop a dataset that, on the one hand, is able to identify the level of respect
for civil liberties and, on the other hand, will be as reliable and valid as possible given
the available resources. In this way, it can serve as an effective tool in the comparison
of individual countries across time and in the comparison of several countries across
space and/or time, either on an intraregional or interregional basis. Whether this goal
has been achieved is – and will remain – an open question, but the decisions have been
followed by explicit justifications. The cards are placed on the table, and other scholars
now have a chance to scrutinize the generative procedures.
Once again, it is worth stressing that the index is the first measure which exclusively focuses on civil liberties understood as personal exertion freedoms. It is different
from, and thus supplements, previous attempts to measure respect for democracy and
personal integrity, respectively. It is an easily reproducible measure of a political regime
aspect: specifically, the upside-down relationship between the rulers and the ruled, and
it builds upon widely shared agreement in liberal theory about fundamental liberties
that people should have the freedom to perform without facing obstructions from the
public authorities.
The design of this dataset has made it possible for scholars to use the scores on the
disaggregate items as they see fit and/or to apply the proposed index. Covering 20 Latin American and 28 post-communist countries on an annual basis from the end 1970s
to 2003, the scope of the dataset is relatively comprehensive. It fits the study it was
initially designed to support, that is, an inquiry into the (dis)respect for civil liberty in
Latin American and post-communist countries after the third wave of democratization.
On the other hand, this agenda also involves a limitation of the number of countries and
years covered, so the utility of the dataset for studies on civil liberty dealing with other
research questions may be restricted – at least until a possible extension is embarked
upon. As the data relate to a rather high level of abstraction, it bears mentioning that
extra contextual and specific information is needed if variations of sub-national level or
detailed accounts for cases are in focus. Nevertheless, the CLD is found to have several
advantages over existing datasets and it is the latest – in some regards even the first,
but hopefully not the last – to address a crucial topic, which has not received much
independent attention in political science.
References
Beetham, David, Sarah Bracking, Iain Kearton and Sturt Weir. 2001. International IDEA Democracy Assessment Handbook. The Hague: Kluwer.
Berlin, Isaiah. 1997. Two Concepts of Liberty. In: Isaiah Berlin. The Proper Study of Mankind:
An Anthology of Essays. London: Chatto & Windus. 191-242.
20
S.-E. Skaaning
Bobbio, Noberto. 1990. Liberalism and Democracy. London: Verso.
Böckenförde, Ernst-Wolfgang. 1991. Staat, Verfassung, Demokratie. Frankfurt a.M.: Suhrkamp.
Bollen, Kenneth. 1993. Liberal Democracy: Validity and Method Factors in Cross-National Measures. In: American Journal of Political Science (37). 1207-1230.
Bollen, Kenneth and Pamela Paxton. 2000. Subjective Measures of Political Democracy. Comparative Political Studies (33). 58-86.
Bowman, Kirk, Fabrice Lehoucq and James Mahoney. 2005. Measuring Political Democracy:
Case Expertise, Data Adequacy, and Central America. Comparative Political Studies (38).
939-970.
Bunce, Valerie. 1995a. Should Transitologists be Grounded? Slavic Review (54). 111-127.
Bunce, Valerie. 1995b. Paper Curtains and Paper Tigers. Slavic Review (54). 979-987.
Case, William. 1996. Can the ‘Halfway House’ Stand? Semidemocracy and Elite Theory in Three
Southeast Asian Countries. Comparative Politics (28). 437-464.
Casper, Gretchen and Claudio Tufis. 2003. Correlation versus Interchangeability: The Limited
Robustness of Empirical Findings on Democracy using Highly Correlated Datasets. Political
Analysis (11). 196-203.
Claude, Richard P. and Thomas B. Jabine. 1992. Exploring Human Rights Issues with Statistics.
In: Thomas B. Jabine and Richard P. Claude (eds.). Human Rights and Statistics: Getting the
Record Straight, Philadelphia: University of Pennsylvania Press. 5-34.
Cingranelli, David L. and David L. Richards. 2004. Cingranelli-Richards (CIRI) Human Rights
Database Coder Manual, http://ciri.binghamton.edu/documentation/web_version_7_31_04_
ciri_coding_guide.pdf (15.02.2006).
Cingranelli, David L. and David L. Richards. 2006. Cingranelli-Richards (CIRI) Human Rights
Database, http://ciri.binghamton.edu/ (04.02.2006).
Collier, David and Steven Levitsky. 1997. Democracy with Adjectives: Conceptual Innovation in
Comparative Research. World Politics (49). 430-451.
Constant, Benjamin. 1988. The Liberty of the Ancients Compared with that of the Moderns. In:
Benjamin Constant. Political Writings. Cambridge: Cambridge University Press. 309-328.
Dahl, Robert. 1971. Polyarchy: Partizipation and Opposition. New Haven: Yale University
Press.
Diamond, Larry. 2002. Thinking About Hybrid Regimes. Journal of Democracy (13). 21-35.
Foweraker, Joe and Roman Krznaric. 2001. How to Construct a Database of Liberal Democratic
Performance. Democratization (8). 1-25.
Freedom House. 2004. Methodology, http://www.freedomhouse.org/template.cfm?page=35
&year=2004 (12.12.2004).
Freedom House. 2006. Freedom in the World, http://www.freedomhouse.org (04.04.2006).
Goertz, Gary. 2006. Social Science Concepts: A User’s Guide. Princeton: Princeton University
Press.
Green, Maria. 2001. What We Talk About When We Talk About Indicators: Current Approaches
to Human Rights Measurement. Human Rights Quarterly (23). 1062-1097.
Habermas, Jürgen. 1992. Faktizität und Geltung. Frankfurt a.M.: Suhrkamp.
Habermas, Jürgen. 1996. Die Einbeziehung des Anderen. Frankfurt a.M.: Suhrkamp.
Hadenius, Axel and Jan Teorell. 2004. Same, Same – But Different: Assessing Alternative Indices of Democracy. Paper presented at the APSA Annual Meeting, Chicago, September 2-5,
2004.
Innes, Judith. 1992. Human Rights Reporting as a Policy Tool: An Examination of the State Department Country Reports. In: T. B. Jabine and R. P. Claude (eds.). Human Rights and Statistics: Getting the Record Straight. Philadelphia: University of Pennsylvania Press. 235-257.
Karl, Terry. 1995. The Hybrid Regimes of Central America. Journal of Democracy (6). 72-86.
Karl, Terry L. and Philippe C. Schmitter. 1995. From an Iron Curtain to a Paper Curtain: Grounding Transitologists or Students of Postcommunism? Slavic Review (54). 965-978.
The Civil Liberty Dataset: Conceptualization and Measurement
21
King, Gary, Robert Keohane and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.
Kitschelt, Herbert. 2001. Post-Communist Economic Reform: Causal Mechanisms and Concomitant Properties. Paper presented at the APSA Annual Meeting, San Francisco, August 29
– September 2, 2001.
Kitschelt, Herbert. 2003. Accounting for Postcommunist Regime Diversity: What Counts as a
Good Cause? In: Grzegorz Ekiert and Stephen E. Hanson (eds.). Capitalism and Democracy
in Central and Eastern Europe: Assessing the Legacy of Communist Rule. Cambridge: Cambridge University Press. 49-86.
Lauth, Hans-Joachim. 2004. Demokratie und Demokratiemessung: Eine konzeptionelle Grundlegung für den interkulturellen Vergleich. Wiesbaden: Verlag für Sozialwissenschaften.
Levitsky, Steven and Lucan Way. 2002. Competitive Authoritarianism. Journal of Democracy
(13). 51-65.
Linz, Juan. 2000. Totalitarian and Authoritarian Regimes. Boulder: Lynne Rienner Publisher.
Marsteintredet, Leiv (2007). Graded Conceptual Membership: Linking Radial Concepts and Fuzzy Sets in the Study of Democracy. The Committee on Concepts and Methods: Political
Concepts Working Paper (19).
McNitt, Andrew. 1988. Some Thoughts on the Systematic Measurement of the Abuse of Human
Rights. In: D. L. Cingranelli (ed.). Human Rights: Theory and Measurement. Houndmills:
Macmillan. 89-103.
Merkel, Wolfgang. 1999. Systemtransformation: Eine Einführung in die Theorie und Empirie der
Transformationsforschung. Opladen: Leske + Budrich.
Merkel, Wolfgang and Aurel Crossaint. 2000. Formale und Informale Institutionen in defekten
Demokratien. Politische Vierteljahresschrift (41). 3-31.
Merkel, Wolfgang, Hans-Jürgen Puhle, Aurel Crossiant, Claudia Eicher and Peter Thiery. 2003.
Defekte Demokratie. Band 1: Theorie. Opladen: Leske + Budrich.
Miller, David. 1991. Introduction. In: David Miller (ed.). Liberty. Oxford: Oxford University
Press. 1-20.
Munck, Gerardo and Jay Verkuilen. 2002. Conceptualizing and Measuring Democracy: Evaluating Alternative Indices. Comparative Political Studies (35). 5-34.
Munck, Gerardo and Jay Verkuilen. 2003. Bringing Measurement Back In: Methodological Foundations of the Electoral Democracy Index. Paper Presented at the APSA Annual Meeting,
Philadelphia, August 28-31, 2003.
Neumann, Franz. 1957. The Democratic and Authoritarian State: Essays in Political and Legal
Theory. Glencoe: Free Press.
Ottaway, Marina. 2003. Democracy Challenged: The Rise of Semi-Authoritarianism. Washington:
Carnegie Endowment for International Peace.
Poe, Steven, Sabine Carey and Tanya Vazquez. 2001. How are These Pictures Different? A Quantitative Comparison of the US State Department and Amnesty International Human Rights
Reports, 1976-1995. Human Rights Quarterly (23). 650-677.
Rawls, John. 1971. A Theory of Justice. Oxford: Oxford University Press.
Saward, Michael. 1994. Democratic Theory and Indices of Democratization. In: D. Beetham
(ed.). Defining and Measuring Democracy. London: Sage. 6-24.
Schedler, Andreas. 2002. The Menu of Manipulation. Journal of Democracy (13). 36-50.
Schmitter Philippe C. and Terry L. Karl. 1994. The Conceptual Travels of Transitologists and
Consodologists: How Far to the East Should They Attempt to Go? Slavic Review (53).
173-185.
Skaaning, Svend-Erik. 2006a. Measuring Civil Liberty. The Committee on Concept and Methods:
Political Concepts Working Paper 8, CIDE.
Skaaning, Svend-Erik. 2006b. Defining and Founding Civil Liberty. CDDRL Working Paper 56,
Stanford University.
22
S.-E. Skaaning
Skaaning, Svend-Erik. 2006c. Democracy besides Elections: An Inquiry into the (Dis)Respect
for Civil Liberty in Latin American and Post-Communist Countries after the Third Wave.
Aarhus: Department of Political Science, University of Aarhus.
Umozurike, U. O. 1998. Human Rights and Development. International Social Science Journal
(158). 535-543.
Waldron, Jeremy. 2003. Security and Liberty: The Image of Balance. Journal of Political Philosophy (11). 191-210.
Zakaria, Fareed. 1997. The Rise of Illiberal Democracy. Foreign Affairs (76). 22-43.
Appendix: Extract from the CLD Codebook
Freedom of Thought, Conscience, and Religion (frerel)
Definition
The component specifies the extent to which individuals and groups have freedom of
thought, conscience, and religion, that is, the right of the citizens to have and change
religion or belief of own choice and alone or in community manifest their religion or
belief in practice, worship, observance, and teaching in private or public as well as
proselytize peacefully without being subject to actual limitations or restrictions.
Component Scale
(1) Severely restricted. Hardly any freedom of religion exists. As a rule, any kind of
religious practice is controlled by the government and harshly suppressed.
(2) Fairly restricted. Some elements of autonomous organized religious practices exist
and are officially recognized, but major religious directions are repressed, prohibited, or systematically disabled.
(3) Modestly restricted. There are minor restraints on the freedom of religion, predominantly limited to a few isolated cases, but as a rule, there are no interventions or
prohibitions on communities or individual worshippers.
(4) Unrestricted. Unhampered freedom of religion exists.
Qualifying Violations
Ad 1) Religious activity is directly prohibited by government policy or is severely restricted for all major religions. In some instances, penalties on persons who engage in
religious practices exist as well as educational campaigns against religion and obliging
citizens to expose believers. Religious leaders are appointed by and subjected to public
authorities, who control the activities of any religious direction in detail. Open expression of certain religious belief or any religious belief in general is incompatible with
membership in the ruling party or attainment of influential public positions.
Ad 2) The government discourages religious beliefs in general or harasses some of
the major religious directions in the country, but traditionally established religions are
The Civil Liberty Dataset: Conceptualization and Measurement
23
recognized and religious practice is generally tolerated, although kept under surveillance by the public authorities. People openly expressing traditional religious beliefs
of certain sorts are discriminated and/or intimidated by public authorities. Sometimes
members of several minor religious groups are not allowed to teach or practice their religion or face severe restrictions concerning registration, places of worship, restrictions
on voluntary conversion, etc.
Ad 3) The government places minor or few restrictions on the rights of religious
groups such as discrimination against a few minority religions in terms of denial of
registration, hindrance of foreign missionaries to enter the country, not allowing citizens or foreigners to proselyte or hindering the access to or construction of places of
worship. Few instances of discrimination and/or intimidation of individuals or groups
due to their (often non-traditional) religion are carried out by public officials.
Ad 4) Citizens enjoy the right to practice any religious belief they choose. Religious
groups may organize, select, and train personnel; solicit and receive contributions; publish works; and engage in consultations without government interference. There are no
government restrictions on establishing and maintaining places of worship. All religious
groups may worship freely and uphold contact with their coreligionists abroad. Active
missionary presence is not restrained. The redistribution of former places of worship
after earlier confiscation is not considered in the assessment. Whether the clergy is able
to freely advocate partisan political views, oppose government laws, support political
candidates, and otherwise freely participate in politics is only taken into consideration
as far as restrictions and repression by public authorities is attributable to religious
rather than direct political discrimination. It does not count as a restriction if religious
communities must register, if the public authorities routinely grant registration and do
not abuse the process to discriminate against a religion, and if the government does not
constrain the right to worship before registration.