Paxton Memo 10/22/09 [email protected] Attention to definition and measurement is critical as social scientists seek to understand the determinants and consequences of liberal democracy and practitioners attempt to spread democracy around the world. Both policy and academic work presupposes the accuracy of measures of democracy. In this memorandum, I address two disparate topics: (1) the use of multiple indicators in measurement, and (2) the appropriate incorporation of gender into measures of democracy. The memorandum largely focuses on conference goals 2, 3, and 5. USE OF MULTIPLE INDICATORS TO ADDRESS RANDOM AND SYSTEMATIC ERROR Using democracy in empirical work requires accurate measurement. But there is potential for both systematic and random error in our measurement of democracy. Random and nonrandom measurement error can bias coefficient estimators of effects and distort comparisons across countries, undermining the empirical results that ignore it. One solution is the judicious use of multiple indicators to measure the theoretical dimensions of democracy. Certainly democracy is a complex, multidimensional concept that is difficult to capture with any one indicator. When we use a measurement strategy that combines individual indicators, it allows better concept/measurement matching, assessment of measurement error, assessment of reliability and validity, and other benefits. In this section I briefly describe the benefits of a multiple indicator strategy for addressing error in measurement. Measurement error is recognized as a problem in all areas of social science, but it is rarely explicitly addressed in the creation of democracy scores. Even random errors in explanatory variables will bias regression coefficients up or down. Knowledge about how to treat random measurement error is widespread but is not always applied. Structural Equation Modeling is perhaps the most common procedure with the capability to take account of random measurement errors and could be quite useful as we work to improve the overall measurement of democracy. Further, when working with subjective measures, models with multiple indicators allow researchers to assess judge­specific effects and biases (Bollen 1993; Bollen & Paxton 1998, 2000). Subjective ratings typically contain both systematic and random measurement errors that can frustrate our attempts to understand democracy. In brief, if the systematic error is related to variables that in turn affect democracy, then results will be confounded with those variables and cannot be corrected simply by applying techniques to handle only random error. If, however, the method factors are idiosyncratic and can be shown to be unrelated to other variables that influence the substantive traits, the systematic error will behave analogous to random measurement error. Thus, especially when subjective measures are used, we must recognize that special care should be taken to determine potential sources of bias and to correct for this bias. Use of multiple indicator models that explicitly account for possible systematic (method) effects, would allow us to develop indices of democracy that have a weaker relation to the method factors than is true of the individual variables that enter the composite. Further, once the presence of method factors is established, we can explore their determinants. Regressing estimated latent method factor scores on variables suspected of contributing to the biases can test the validity of these concerns. Such variables, if significant, contribute to the "bias" in the substantive concept and can be corrected during measurement or analysis. INCORPORATING WOMEN’S SUFFRAGE AND REPRESENTATION INTO MEASURES OF DEMOCRACY Although one can argue that the concept of democracy is gendered in principle, women are not typically included in practice. That is, while definitions are generally inclusive, requiring all adults of a certain geographic area to have certain political privileges, measures of democracy often fail to include women as political participants (Paxton 2000; 2008). The omission of women is quite obvious in some cases, for example when a measure of democracy explicitly uses male suffrage as the sole indicator of a country’s transition to democracy (e.g., Huntington 1991; Rueschemeyer, Stephens and Stephens 1992). It can be harder to detect in graded measures, where the achievement of female suffrage does not change a country’s score on a measure of democracy (e.g., Polity IV, Jaggers and Gurr 1995). The omission of women is an ongoing issue in the measurement of democracy, affecting both classic studies (Lipset 1959) and more recent work (Alvarez et al 1996; Mainwaring et al. 2001; Reich 2002). The omission of women from measures of democracy is not often recognized because, on the surface, gender is not an issue. Theorists use gender­neutral language when defining democracy and measure democracy with seemingly universal concepts such as the people’s right to vote. But the appearance of ‘neutrality’ toward gender in political theory or ‘equality’ between men and women in government actually hides substantial gender inequality (Pateman 1989; Phillips 1991; Young 1990). If gender­neutral language is used in principle, but in practice only men appear, then women are not equal in our theories or measures, but invisible. Fundamentally the problem is one of operationalization. Although definitions of democracy stress the importance of universal suffrage and participation of all major social groups (Dahl 1971; Diamond, Linz, and Lipset 1990; Schmitter and Karl 1991; Bollen 1990), in measures of democracy women are often deliberately excluded or simply overlooked. Further, the extent of women’s representation (or the representation of other underrepresented groups) is never considered in our measures of democracy. A reevaluation of the conceptualization and measurement of democracy by APSA affords the opportunity to rectify this continuing problem. Solutions can take one of two forms, one minimal, one more radical. The minimal solution to the problem of the omission of women (and other political minorities) from our measures of democracy is simply to require that our measures match our definitions. Almost all definitions of democracy derive from Dahl’s (1971:4) classic distinction between contestation/competition and participation/inclusion and the second dimension, participation, is central to most definitions of democracy. Dahl (1971:2) argues that a democratic regime is “completely or almost completely responsive to all its citizens.” Following this lead, contemporary scholars’ definitions of democracy typically involve some discussion of universal suffrage. Across definitions, democratic participation is seen to include “adults” (Schmitter and Karl 1991:77), “the people” (Vanhanen 2000:252), and the “nonelite” (Bollen 1990:9). This criterion implies, though it does not directly state, that women should be included. As a major social group, comprising 50% of a typical adult population, women are clearly part of a universal suffrage requirement. The full incorporation of women and other political minorities into the measurement of democracy does require explicit attention to the participation dimension of democracy and poses a challenge to procedural definitions and measures of democracy (Schumpeter 1942; Przeworski et al. 2000). It also requires some revision of existing measures of democracy (e.g., Polity, ACLP) and a commitment to more inclusive measurement in the future (e.g., Bowman, Lehoucq, and Mahoney 2005). Treating democracy as a dichotomy can make the incorporation of women and other minorities particularly difficult. Graded measures allow more flexibility. Useful also are adjectives to describe types of democracy (Collier and Levitsky 1997). As an example, compare the coding of the United States transition to democracy across various studies. Muller (1988), Huntington (1991), and Rueschemeyer, Stephens and Stephens (1992) each provide different dates for the transition to democracy in the United States. Huntington places the U.S. as a democracy in 1828, while Muller claims it wasn’t a democracy until 1865. Rueschemeyer, Stephens and Stephens do not score the U.S. as a full democracy until 1965. These different dates create a 137­year measurement difference, based largely on an inconsistent operationalization of suffrage. The confusion is erased if we think of transitions to democracy in continuous terms—
the United States was simply at different stages of democracy on those dates. In 1828, property requirements were relaxed, in 1865, universal male suffrage was instituted, in 1920, female suffrage was introduced, and in 1965, nearly universal suffrage with no literacy requirements was achieved. An important benefit of implementing this minimal fix would be improved longitudinal consistency in our measurement. Although some authors such as Samuel Huntington or Scott Mainwaring have argued for retrospective standards, there are at least two problems with allowing the definition of democracy to vary across time periods without acknowledging a diminished subtype.1 First, classification of relevant time periods could very widely across scholars, introducing inconsistency. Second, retrospective standards disallow longitudinal research and an ability to track the growth in democracy over time. If democracy is defined, and therefore measured, in terms relative to a particular time period, then researchers can only compare across countries at that given time point. Of course, scholars must recognize that our theories of democracy are likely to be very different if we use a constant measure of democracy across time, rather than a measure with different criteria for early, middle, and later periods. Apart from correcting the most egregious problems of gender omission in our measures, we could also opt for a more radical reimagining of the measurement of democracy by fully incorporating Dahl’s participation dimension through the inclusion of descriptive representation. Although not often recognized in mainstream studies of democracy, feminist theorists have made compelling arguments for a fuller inclusion of women in our understanding of democracy (Pateman 1989; Phillips 1991; Waylen 1994; Paxton 2000). The case for descriptive representation hinges on the notion that racial, ethnic, and gender groups are uniquely suited to 1
The recommendation to define countries as democratic based on the standards of the era is typically made to avoid the problems incurred by the lack of female suffrage in countries declared to be democratic before World War II. Allowing retrospective definition/measurement avoids overturning the democratic credentials of countries such as Switzerland that do not meet contemporary definitions of democracy in earlier periods (e.g., Switzerland does not grant women the right to vote until 1971). represent themselves in democracies. In principle, democratic ideals suggest that elected representatives will serve the interests of the entire community and be able to transcend any specific interests based on their own characteristics such as sex, race, or age. But in practice, ‘while we may all be capable of that imaginative leap that takes us beyond our own situation, history indicates that we do this very partially, if at all’ (Phillips 1991: 65). Because social groups (gender groups, ethnic groups) have different interests due to varied economic circumstances or histories of oppression, representation by groups other than ones own is not assured. Thus, we may need to think more fully about the concept of representation in our measures of democracy. Certainly the de jure right to participate is quite different from de facto informal restrictions that limit actual use of the right to vote or run by women or other groups. Put another way, if certain segments of the population are not included as representatives does it suggest continuing discrimination that makes a country less democratic? Taking representation seriously has the potential to move us toward the measurement of descriptive as well as formal participation in our measures of democracy. Here, it may be useful to think of “adjective” modifications to the concept of democracy, perhaps making a distinction between “formal democracy” and “descriptive democracy,” much as is seen in the representation literature. A benefit to the more radical incorporation of gender and minority representation into our measures of democracy is that it would help scholars make distinctions between countries with similar democracy scores. Many scholars have lamented that democracy indicators tend to be insensitive to important gradations in the quality of democracy, meaning that “Papua New Guinea and Sweden receive the same score (“democratic”), despite evident difference in the quality of elections, civil liberties, and barriers to competition…” (Gerring and Coppedge 2009). Including direct measures of Dahl’s participation dimension, such as the representation of women in parliaments, can help make more nuanced distinctions among democracies. Regardless how it is accomplished, including women will ensure that researchers’ measures of democracy match their definitions. When democratic scholars agree to accept definitions of democracy that are inclusive, they also must be prepared to accept the consequences of that definition for their measurement of democracy and its conclusions about the history and causes of democracy. Continuing to write women out of our measures of democracy as they represent an inconvenience to existing theory, measures, or explanations, does a true disservice to our ultimate understanding of democracy. REFERENCES Alvarez, Mike, Jose Antonio Cheibub, Fernando Limongi and Adam Przeworski. 1996. “Classifying Political Regimes.” Studies in Comparative International Development 31:2­36. Bollen, Kenneth A. and Pamela Paxton. 1998. “Detection and Determinants of Bias in Subjective Measures.” American Sociological Review 63:465­478. Bollen, Kenneth A. and Pamela Paxton. 2000. “Subjective Measures of Liberal Democracy.” Comparative Political Studies 33:58­86. Bollen, Kenneth A. 1990. “Political Democracy: Conceptual and Measurement Traps.” Studies in Comparative International Development 25: 7­24. Bollen, Kenneth A. 1993. “ Liberal democracy: Validity and method factors in cross­national measures.” American Journal of Political Science 37:1207­1230. Bowman, Kirk, Fabrice Lehoucq, and James Mahoney. 2005. “Measuring Political Democracy:Case Expertise, Data Adequacy, and Central America.” Comparative Political Studies 38:939­970. Collier, David and Steven Levitsky. 1997. “Democracy with Adjectives: Conceptual Innovation in Comparative Research.” World Politics 49 (April): 430­51. Dahl, Robert A. 1971. Polyarchy: Participation and Opposition. New Haven: Yale University Press. Diamond, L., Linz, J. J. and Lipset, S. M. (eds.) 1990. Politics in Developing Countries: Comparing Experiences with Democracy. Boulder, CO: Lynne Rienner. Huntington, Samuel. 1991. The Third Wave: Democratization in the late 20th Century Norman, OK: University of Oklahoma Press. Jaggers, Keith, & Gurr, Ted Robert. 1995. “Tracking democracy’s third wave with the Polity III data.” Journal of Peace Research 32:469­482 Lipset, Seymour Martin. 1959. “Some Social Requisites of Democracy: Economic Development and Political Legitimacy.” American Political Science Review 53:69­105. Mainwaring, Scott, Daniel Brinks, and Aníbal Pérez­Liñán. 2000. “Classifying Political Regimes in Latin America, 1945–1999.” Studies in Comparative International Development 36:37­65. Muller, Edward. 1988. “Democracy, Economic Development and Income Inequality” American Sociological Review 53:50­68. Pateman, Carole. 1989. The Disorder of Women: Democracy, Feminism, and Political Theory. Stanford: Stanford University Press. Paxton, Pamela. 2000. “Women’s Suffrage in the Measurement of Democracy: Problems of Operationalization.” Studies in Comparative International Development 35:92­111. Paxton, Pamela. 2008. “Gendering Democracy.” Pp. 47­70 in Politics, Gender, and Concepts: Theory and Methodology, edited by Gary Goetz and Amy Mazur. Cambridge: Cambridge University Press. Phillips, A. 1991. Engendering Democracy. University Park, Pennsylvania: Pennsylvania State University Press. Przeworski, A., Alvarez, M., Chiebub, A., & Limongi, F. 2000. Democracy and development: Political institutions and well­being in the world:1950­1990. Cambridge: Cambridge University Press. Rueschemeyer, Dietrich, Evelyne Huber Stephens and John D. Stephens. 1992. Capitalist Development and Democracy. Chicago: University of Chicago Press. Schmitter, Philippe C. and Terry Lynn Karl. 1991. “What Democracy is…and is Not.” Journal of Democracy 2 (Summer): 75­88. Schumpeter, Joseph. 1942. Capitalism, Socialism and Democracy. New York: Harper and Row. Vanhanen, Tatu. 2000. “A new dataset for measuring democracy, 1810­1998.” Journal of Peace Research 37:251­265. Waylen, G. 1994. “Women and Democratization: Conceptualizing Gender Relations in Transition.” World Politics 46:327­354 Young, I. M. 1990. Justice and the Politics of Difference. Princeton, NJ: Princeton University Press.