6. Left-right party ideology and government policies: A
meta-analysis
Kurs Public Choice
WS 2009/10
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
1
6. Left-right party ideology and government policies:
A meta-analysis
1. Introduction
2. Data and Methods
3. Empirical Results
4. Overview and Discussion
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
2
1. Introduction
(1) The question of the impact of political parties on policy has been
the object of much research and debate in the past decades. The
main theoretical debate concerns about the relative weight of
socio-economic and political variables in determining policy
outputs.
(2) Advocates of the so-called ‘convergence’ school of thought argue
that industrialized societies of the twentieth century have become
increasingly similar, facing the same kind of problems and
applying the same kind of solutions.
(3) Consequently, the argument goes, political, institutional and
cultural differences do not matter much when it comes to
explaining variations in policy outputs.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
3
1. Introduction
(4) In reaction to the convergence argument, policy scholars have
suggested that indeed politics does matter. Without denying the
importance of economic factors, advocates of the ‘politics matters’
school of thought argue that there is a correlation between
partisan variables and policy outputs.
(5) Much empirical research on the policy impact of political parties
has been published in the form of comparative analyses based on
precise quantitative definitions and measurements within the wellestablished scientific framework of partisan theory (Hibbs 1977,
1992).
(6) Ceteris paribus, changes in the left-right party composition of
government are hypothesized to be related to changes in policy.
This is expressed by a general equation of the form:
Policy outputs = f (left-right variations in the party composition of
government, variations in controlling factors).
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
4
1. Introduction
(7) This general equation has been applied, in a large number of
studies comparatively to measure partisan influence on
government actions in a variety of policy domains across different
countries.
(8) The quantitative literature on partisan theory displays a high level
of accumulation but it has not achieved a comparable level of
scientific aggregation. By accumulation one means that one has a
large literature (with publications numbering in the hundreds).
(9) The goal in this lecture is to contribute to this debate by
summarizing how the partisan influence literature assesses the
relationship between the leftright party composition of
government and changes in public policy through a meta-analysis
of empirical tests of partisan theory in OECD countries.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
5
1. Introduction
(10)A meta-analysis is an analysis of analyses. It is the statistical
analysis of a large collection of test results coming from individual
studies for the purpose of integrating the findings:
i. In meta-analysis, ‘the findings of multiple studies should be
regarded as a complex data set, no more comprehensible without
statistical analysis than would be hundreds of data points in one
study’.
ii. The statistical methods used to accomplish this task are similar to
the ones used in primary analyses of the original data.
The major difference is that each case in a meta-analysis is an
estimate obtained from the original data whereas each case in a
primary analysis is obtained directly from the original data.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
6
1. Introduction
iii. A meta-analysis is useful not only to draw out general patterns
from a wide variety of empirical sources but also because it
enables researchers to resolve disputes in the literature, to
determine which factors have contributed to systematic
differences across studies, and to identify areas that have been
neglected and warrant further investigation.
iv. The meta-analysis helps identify both substantive influences on
research findings and artefacts resulting from particular research
methods and data analysis techniques.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
7
2. Data and methods
(1)Although they share a common scholarly purpose,
meta-analyses (MAs) and literature reviews (LRs) are
quite different from each other.
(2)LRs examine how a particular phenomenon has been
studied by scholars in published works.
(3)MAs deal with specific empirical hypotheses and try to
investigate, on the basis of the empirical work so far
published, whether there is an impact of some
independent variables upon dependent variables, and,
if so, what the magnitude of this impact is.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
8
2. Data and methods
(4) Like any other empirical research, MAs are vulnerable to various
threats to validity:
i. The main threat to validity in this MA is heterogeneity among the
studies incorporated in the analysis.
ii. One tries to respond to this threat to validity;
first, by establishing explicit criteria for the selection of individual
studies in order to ensure that the tests within each study are
comparable; and second, by coding relevant published
information in the form of variables most susceptible to
influencing the predictive performance of each test.
iii. The criteria for the selection of cases and the rules for
constructing the variables in the MA were organized in a
codebook which served as guidance for researchers to identify
individual parameter estimates and their relevant attributes in
each study.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
9
2.1. Selection of cases
(1) One concern in selecting individual cases for the meta-analysis
was comprehensiveness and/or comparability.
To have confidence in measures, one needs to be sure that the
selection process was representative of the available material.
To achieve this goal, one tries to minimize discretion in selection
by considering everything we could find in the published record.
(2) Over 600 studies (mainly journal articles and book chapters) were
identified, containing empirical results.
(3) In this MA we are interested in reporting cross-national
comparisons and include results from cross-sectional studies and
times-series cross-sectional studies only.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
10
2.1. Selection of cases
(4)The validity of the meta-analysis may also be
threatened by the wide variety of measures of party
ideology used by partisan theorists to explain policy
outputs in individual studies (the explanans).
To limit heterogeneity in party ideology variables, one
considers only tests in which ideology is defined as
either left or right.
(5)Another possible threat to validity in MA is the lack of
sample homogeneity in the estimation methods.
One of the objectives of this MA is to extract overall
measures of the relationship between the party
composition of government and policy outputs from
single coefficients found in individual studies.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
11
2.2. Factors in the meta-analysis
(1) The dependent variable in this MA is the predictive performance
of individual tests of partisan theory, measured on the basis of
parameter estimates of party impact.
(2) These estimates are operationalized in terms of three attributes:
(i) estimates that support the partisan theory hypothesis (the
coefficient is significant and in the predicted direction) are
recorded as ‘successes’;
(ii) estimates that fail to confirm the hypothesis (the coefficient fails
the test of statistical significance) are reported as ‘failures’;
(iii) estimates that contradict the hypothesis (the coefficient is
significant and in the wrong direction) are reported as
‘anomalies’.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
12
2.2. Factors in the meta-analysis
(3) Individual tests of partisan impact on welfare policies typically
postulate that, ceteris paribus, welfare intervention by the state is
positively correlated with the presence of leftist parties in
government and negatively correlated with the presence of rightist
parties in government (denoted L+, R−).
(4) The approach postulates that spending on development assistance
is positively correlated with the presence of the left in government
and negatively correlated with the presence of the right in
government (L+, R−).
(5) The hypothesis is reversed for military spending and military
alliance membership: military spending and state involvement in
military alliances correlates positively with the presence of the right
in government and negatively with the presence of the left in
government (L−, R+).
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
13
2.2. Factors in the meta-analysis
(6) One important variant of this approach is the political businesscycle hypothesis, which states that the standard relationship (L+,
R−) holds for state intervention in macro-economic policy, public
debt as a percentage of GDP, and progressivity of taxes. The
reversed relationship (L−, R+) holds for privatisation, and budget
balance as a percentage of GDP.
(7) A final approach examines how partisan factors affect the overall
size of government. The approach postulates that, ceteris paribus,
government size correlates positively with the presence of the left in
government and negatively with the presence of the right in
government (L+, R−).
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
14
2.2. Factors in the meta-analysis
(8) Individual tests of partisan theory also vary in terms of how they
operationalize observed government activity. Most studies
operationalize government activity in terms of public spending or
revenues from taxes (often expressed as a percentage of GDP).
(9) In order to assess the impact of variation in the definition of
policy domains on our conclusions, we record whether individual
parameter estimates apply
i.
ii.
iii.
iv.
to social welfare policies,
the size of the state,
economic policies, and
or foreign and defense policies.
We also record whether the measure of government activity in each
test is based on
a) financial data (e.g., public spending) or
b) non-financial data (e.g. regulation or administrative action).
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
15
2.2. Factors in the meta-analysis
(10)Individual authors often calculate the presence of the left or the
right in government in the form of sophisticated indices. However,
so far we do not have a standardized normalized measure.
(11)There is some disagreement about the range of parties to be
included in the measure. Most studies examined in this MA report
the presence of the left and/or the right in government on the basis
of either leftist parties, a left-right scale, conservative parties, or
major parties of the right.
(12)Individual measures of the party composition of government also
differ in their definitions of relative party strength. The most
frequently used measure is the party composition of the cabinet.
(13)This is justified on the grounds that in advanced liberal
democracies, political power is basically exercised through the
cabinet and that governmental policy is directly related to the
allocation of ministerial portfolios among different parties.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
16
2.2. Factors in the meta-analysis
(14)Another factor that may affect the validity of our meta-analysis is
the variation in the specification of the model in each individual
study, i.e. the number and nature of explanatory or control
variables that partisan theorists explicitly include in their datasets.
(15)There is evidence that the statistical significance and the direction
of estimates of partisan impact may be affected by institutional
factors such as
the division of power between parties in and out of government,
the frequency of electoral competition, or the time a government
has been in power.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
17
3. Empirical Results
(1) Out of over 600 studies identified in the bibliographic search, 43
proved to correspond to our selection criteria.
(2) The studies are listed in chronological order in Table 3 along with
information
(i) about the period,
(ii) the number of countries over which the tests were run,
(iii) the relevant policy domain, and
(iv) the left-right attribute used to define the party composition of
government variable.
(3) Table 1 reports only the summary information for each study.
Individual tests within each study may involve different time
periods, countries, policy domains and different measures of party
ideology.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
18
Table 1: Characteristics of quantitative studies on the ideology-policy relationship
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
19
Table 1: Characteristics of quantitative studies on the ideology-policy relationship (cont.)
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
20
3.1. Overall left-right partisan effects
(1) Table 1 also presents the number of relevant tests in each study
and the frequencies with which these tests are reported
as ‘successes’ (a coefficient associated with a right or a left
partisan variable is significant (p < 0.10) and in the correct
direction),
‘anomalies’ (a coefficient associated with the right or the left is
significant and in the opposite direction to the one hypothesised)
or ‘failures’ (a non-significant coefficient) in this study.
(2) Once the results of each individual test have been sorted into
successes, failures and anomalies, we can use the simple ‘votecounting’ meta-analytic procedure to assess the overall
relationship between the left-right party composition of
government (the explanans) and policy outputs (the
explanandum).
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
21
3.1. Overall left-right partisan effects
(3) Looking at the overall distribution of tests (bottom row of Table
1), we see that, out of the 693 tests, only 154 (22 percent) are
reported as successes and 48 tests (7 percent) are reported as
anomalies.
The remaining 491 tests (71 percent) fail the statistical significance
test (at 10 percent) and are reported as failures.
(4) The success rates appear remarkably stable over time:
86 out of 385 tests published in or before 1986 are reported as
successes (22 percent);
68 out of 308 tests published after 1987 are reported as successes
(22 percent).
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
22
3.2. Mediating factors
(1) Clearly, these numbers cannot be interpreted as evidence in
support of the left-right party impact hypothesis. Applying the votecounting technique to the data suggests that, since the ‘failure’
category is the modal category, we cannot reject the null
(convergence) hypothesis of no party impact based on the average
predictive power of individual tests.
(2) Table 2 presents a breakdown of individual tests by policy domain.
We see from the table that 69 (24 percent) out of 288 tests in the
welfare policy domain are reported as successes.
(3) The difference in predictive performance across specific areas
within the welfare domain (from a low of 19 percent to a high of
27 percent) is not statistically significant. Next we see that 59 (37
percent) out of 161 tests of the size of the state are reported as
successes.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
23
Table 2: Success rate of individual tests by policy domain
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
24
3.2. Mediating factors
(3) As in the welfare domain, the deviations from the mean in the size
of the state domain are not statistically significant.
(4) The overall success rates are much lower in the two remaining
policy domains. There are 20 successes (13 percent) out of 148
tests in the foreign affairs and defence policy domain and only 6
successes (6 percent) out of 96 tests in the economic policy domain.
The deviations from the mean success rate are statistically
significant in both the foreign affairs and the economic policy
domains.
(5) Tests of left-right party impact on the level of state intervention in
the economy have a (relatively) very high degree of success (39
percent). On the other hand, the success rate in the public budget
and taxation areas is virtually zero.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
25
Table 3: Success rate of individual tests across other variables
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
26
3.2. Mediating factors
(6)We see from Table 3 that there are twice as many tests
based on the presence of the left (472) as compared
with tests based on the presence of the right (221) in our
sample.
(7)Tests that report the presence of the right generate a
higher rate of success on average (25 percent) than tests
that report the presence of the left (21 percent).
(8)Looking at the party strength variable of Table 3, we
see that using the percentage of cabinet portfolios,
although the preferred method of calculating party
strength, has no statistical impact on success when
compared with alternative methods.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
27
3.2. Mediating factors
(9) Next, we see that 79 (28 percent) out of 279 multivariate tests are
reported as successes against 75 (18 percent) out of 414 bivariate
tests.
(10)The paired difference for the model specification variable is
statistically significant, as is the paired difference for sample size
(the proportion of success is 20 percent for smaller samples against
28 percent for larger samples) and for the measure of government
activity (the rate of success is 23 percent for financial data
compared with 12 percent for non-financial data).
(11)The paired difference for the historical period variable is also
statistically significant: 92 (27 percent) out of 339 tests based on
the post-1973 period are reported as successes against 64 (18
percent) out of 354 tests based on the pre-1973 period.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
28
Table 4: Success rate of individual tests by historical period and by
policy domain
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
29
3.2. Mediating factors
(12)An interesting pattern emerges when we crosstabulate
the historical period variable with policy domains
(Table 4).
(13)Post-1973 tests of party impact on the size of the state,
foreign affairs and economic policy have significantly
higher average success rates than pre-1973 tests of
party impact in these policy domains.
(14)However, individual tests of party impact on welfare
policy follow the opposite direction.
The average success rate of estimates of party impact
on welfare is smaller for the post-1973 period than for
the pre-1973 period.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
30
3.2. Mediating factors
(15)Table 5 presents a multivariate logistic regression model in which
the dependent variable takes the value of 1 when a test is reported
as a success and 0 otherwise.
There were too few anomaly cases to include them as a distinct
variable in this analysis.
(16)Three policy domain variables are included in the model to assess
the predictive performance of tests of party impact on the size of
the state (total public spending and taxation), foreign affairs, and
the economy as compared with the predictive performance of tests
of party impact on welfare (the reference policy domain).
(17)The next two variables in the model are dichotomous variables
intended to assess whether the predictive performance of
individual tests is affected by the use of non-financial policy data
(as opposed to financial policy data) and by the inclusion of a left
(instead of right) partisan variable.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
31
Table 5: Multivariate logit analysis of individual tests’ predictive
performance
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
32
3.2. Mediating factors
(18)From Table 5, we see that, ceteris paribus, the success rate of
individual tests is not significantly affected by how partisan impact
is operationalized.
(19)Whether individual tests use the left or the right to measure
partisan impacts does not affect predictive performance, and
neither does the definition of party strength (although measures of
party strength based on the percent of popular votes perform
better than measures based on cabinet portfolio allocation).
(20)The data also rule out sample size and the use of financial (as
opposed to non-financial) data as causes of variation in predictive
performance.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
33
3.2. Mediating factors
(21)Thus, keeping the other variables in the model constant, the
predicted logit of success for a multivariate, post-1973 test of
the left-right division of the popular vote on the size of the state
is L = −1.17 + 0.56 + 0.67 + 0.50 + 0.67 = 1.23.
(22)The predicted probability of success for such a test is therefore
P = 1/[1 + e−(1.23)] = 0.77.
By comparison, the probability of success of a bivariate, pre1973 test of partisan impact on welfare using a mixed measure
of party strength is P = 1/[1+e−(−1.17)] = 0.24.
(23)This demonstrates that there are clearly identifiable conditions
under which the probability of support for the left-right party
influence hypothesis can be substantially increased.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
34
4. Overview and discussion
(1) We started this review of 693 published cross-section estimates of
left-right party impact on policy with a series of questions that
have not been addressed in a meta-analytic format before:
i. how often do estimates support the left-right party impact
hypothesis?
ii. What is the average magnitude of the effect size of left-right party
impact?
iii. What (if any) are the determinants of variations in statistical
support?
iv. Are partisan cycle explanations more sensitive to substantive
factors (e.g., the policy domain of government intervention; the
historical period of analysis) or to issues of measurement and
methodology?
v. How do our results contribute to the advancement of knowledge in
the field?
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
35
4. Overview and discussion
In response to the first two questions:
154 (22 percent) estimates support the left-right party impact
hypothesis,
48 (7 percent) estimates contradict the hypothesis, and
491 (71 percent) estimates fail to support the hypothesis.
Applying the ‘vote-counting’ meta-analytic technique to these
numbers leads us to conclude that the null hypothesis of no left-right
party impact cannot be rejected.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
36
4. Overview and discussion
We reach a similar diagnosis when we use a revised ‘combined tests’
technique to generate an ‘average effect size’ of left-right partisan
impact on policy.
The 0.28 average correlational effect between party and policy
indicates that approximately 8 percent of the variation in policy
output is accounted for by the variation in the left-right party
composition of Government.
Literature:
Louis M. Imbeau, Francois Pétry and Moktar Lamari (2001), LeftRight Party Ideology and Government Policies: A-Meta-Analysis,
European Journal of Political Research 40/1, 2001, pp. 1-29.
WS 09/10
© Prof. Dr. Friedrich Schneider, University of Linz, Austria
37