How Partisan Conflict in Congress Affects Public Opinion:
Strategies, Outcomes, and Issue Differences
D.J. Flynn
PhD Candidate, Department of Political Science
Northwestern University
[email protected]
Laurel Harbridge*
Assistant Professor, Department of Political Science
Faculty Fellow, Institute for Policy Research
Northwestern University
[email protected]
Scholars are increasingly interested in how partisan conflict in Congress affects public
evaluations of institutional performance. Yet, existing research overlooks how the public
responds to one of the most widely discussed consequences of partisan conflict:
legislative gridlock. We develop expectations about how partisan conflict resulting in
partisan wins, losses, and gridlock will affect evaluations of Congress, and how these
relationships will differ across consensus and non-consensus issues. Results from two
survey experiments indicate that partisan conflict resulting in a victory for one’s own
party boosts approval relative to compromise, but conflict resulting in gridlock
substantially damages approval. However, the degree to which gridlock decreases
approval hinges on the type of policy under consideration. On consensus issues, citizens
reward legislative action by either party—their party or the opposing party—over
gridlock.
*
Corresponding author. The authors would like to thank James Curry, Jamie Druckman,
Patrick Egan, Yanna Krupnikov, Jennifer Jerit, Thomas Leeper, Neil Malhotra, Kevin
Mullinix, and Jennifer Wolak for comments on a previous draft. The supplemental
appendix for this paper will be posted on the author’s website. Replication files will be
posted on the Open Science Framework.
How Partisan Conflict in Congress Affects Public Opinion: Strategies, Outcomes, and
Issue Differences
To what extent does partisan conflict in Congress affect public evaluations of
institutional performance? In recent years, historic levels of party polarization have
coincided with some of the lowest congressional approval ratings in decades (McCarty,
Poole, & Rosenthal, 2006; Pew, 2015; Theriault, 2008). There are several reasons to
think that heightened party conflict would have significant effects on approval. In
general, a large body of research suggests that governing strategies – and resulting policy
outputs – influence public attitudes toward and the perceived legitimacy of political
institutions (e.g., Doherty, 2015a; Hibbing & Theiss-Morse, 2002). Partisan conflict in
particular plays a central role in governing strategies, affecting both legislative processes
and policy outputs. Thus, a growing body of literature has sought to isolate when and
why partisan conflict increases or decreases evaluations of Congress, and whether the
public prefers bipartisan compromise to partisan conflict (Durr, Martin, & Wolbrecht,
1997; Harbridge & Malhotra, 2011; Harbridge, Malhotra, & Harrison, 2014; Jones, 2013;
Ramirez, 2009).
Although illuminating in many regards, these studies tell us little about how the
public responds to one of the most widely discussed consequences of partisan conflict:
legislative gridlock (see Mann & Ornstein, 2012). Extant studies of partisan conflict and
public opinion often overlook the outcome of conflict (Durr et al., 1997; Ramirez, 2009),
or explicitly focus on partisan victories (Harbridge et al., 2014). This gap is all the more
surprising given recent public and scholarly concern about partisan conflict and
legislative productivity (e.g., Binder, 2014; Jones, 2001; Mann & Ornstein, 2006, 2012).
In this article, we address this gap by examining how the public responds when partisan
1
conflict results in legislative gridlock.1
We argue that the key to understanding the complex relationship between partisan
conflict and public opinion lies in considering the implications of partisan conflict for
legislative outcomes, in particular whether conflict results in gridlock. When parties
eschew compromise, partisan conflict can result in a win for one’s own party, a win for
the opposing party, or gridlock. Specifically, we suggest that while citizens approve of
partisan conflict when it results in a win for their party, they should disapprove when it
prevents Congress from performing its basic responsibilities to address national problems
(Adler & Wilkerson, 2013; Butler & Powell, 2014; Hibbing & Theiss-Morse, 1995).
Evaluations of Congress may not only be lower following gridlock than when
partisanship results in a win for one’s own side, but on issues where the parties disagree
over the means but agree on the end goals of policy (i.e., consensus issues), gridlock may
be even worse than a win for the opposing side. On more contentious issues, gridlock
may still damage congressional evaluations, but may be viewed more similarly to a
victory by the opposing party. We test these expectations with two survey experiments in
which we manipulate the consequences of party conflict for legislative outcomes. Our
approach varies not just the legislative behavior of the parties (i.e., compromise or
1
Our use of the term “gridlock” captures popular understanding of the term: failure to
pass legislation on a particular issue (for any reason related to partisan conflict). As such,
the term includes instances where individuals with pivotal positions in the institution
oppose a proposal over the status quo (Krehbiel, 1998), but also instances where parties
refuse to accept compromises in order to score political victories (Gilmour, 1995).
2
partisanship), but also the consequence of partisanship (i.e., a partisan win, partisan loss,
or gridlock). As a result, this work speaks to larger questions regarding the relationship
between approaches to governing, policy outputs, institutional approval, and legitimacy
(Gibson, Caldeira, & Spence, 2005; Hibbing & Theiss-More, 2001; Tyler, 1994).
We make three novel contributions to the literature on partisan conflict and public
opinion toward Congress. First, we test when party conflict is attractive by comparing
evaluations of Congressional performance across different legislative strategies –
partisanship versus compromise – and outcomes of partisanship – win for one’s own
party, win for the opposing party, and gridlock. Second, we investigate whether the effect
of gridlock on public opinion depends on whether gridlock is framed as resulting from
ideological disagreement versus strategic partisan considerations (e.g., elections). The
media regularly invoke both ideological and strategic partisan frames in their coverage of
politics (Cappella & Jamieson, 1997; Lawrence, 2000), and this distinction may be
important for understanding how the public reacts to legislative gridlock. Third, we
consider how a critical issue-level factor—the degree of cross-party consensus over
policy goals—affects public responses to gridlock. The opposition to gridlock may be
much greater when consensus exists on policy goals (even as parties disagree on the
means) than when parties disagree over both the means and the goals of policy.
Combined, our approach sheds light on the complex relationship between partisan
conflict and public opinion, and highlights how public evaluations of Congress rest on
considerations that go beyond policy congruence.
The experimental results indicate that citizens approve of how Congress is
handling policymaking when partisan conflict produces a win for one’s own party.
3
However, we also uncover evidence that citizens disapprove when partisan conflict
prevents Congress from acting on an important national issue. In fact, on a consensus
issue, partisans are more approving of Congress’ handling of policymaking when a policy
debate results in a win for the other party than when the debate ends in gridlock. This
surprising finding runs counter to the prevailing view of partisans in the mass public as
hostile to members of and ideas from the opposing party (e.g., Iyengar, Sood, & Lelkes,
2012). While some research suggests that citizens may endorse a range of policy
proposals to deal with national problems (Egan, 2014), that work has failed to link
proposals to particular parties (e.g., “Party A suggests solution a, Party B suggests
solution b”)—arguably the most common format in which policy proposals are described
in the real world. We address this gap and find that people still value action by the
opposing party on important issues. These results are striking in light of research on
partisan cues, which suggests that citizens should move overwhelmingly in the direction
of their party’s position (Druckman, Peterson, & Slothuus, 2013; Slothuus & de Vreese,
2010). Finally, while both forms of gridlock – ideological and partisan – result in lower
evaluations of Congress, the evidence is suggestive that approval is lowest when gridlock
is attributed to strategic partisan behavior. Citizens are significantly more accepting of
legislative inaction when it is characterized as the result of genuine ideological
disagreements between the two parties.
Background and Expectations
National polls regularly uncover widespread support for bipartisan cooperation in
Congress. For instance, Pew reports that eight-in-ten Americans agree with the statement
“I like political leaders who are willing to make compromises in order to get the job
4
done” (Pew, 2012). Likewise, six-in-ten respondents prefer that the majority in Congress
tries to pass legislation with bipartisan support as opposed to passing legislation without
minority support (CBS News, 2009). Recent increases in party polarization (e.g.,
McCarty et al., 2006; Theriault, 2008), resulting in both partisan victories and legislative
gridlock (Binder, 2003, 2014; Burden, 2011; Jones, 2001), stand in stark contrast to this
expectation. Driven in part by this contradiction, in recent years scholars have begun to
reexamine legislators’ incentives for bipartisanship, questioning whether citizens actually
reward compromise over partisan conflict. For instance, recent work has considered how
voters respond to moderation versus ideological extremity (Canes-Wrone, Brady, &
Cogan, 2002) and independence versus party loyalty (Carson, Koger, Lebo, & Young,
2010; Harbridge & Malhotra, 2011) in roll-call voting, and to civil versus uncivil
discourse in politics more broadly (Brooks & Geer, 2007; Fridkin & Kenney, 2011; Mutz
& Reeves, 2005; Paris, 2015).
These questions are closely related to a longstanding body of literature that
connects preferences over styles of governance, policy outputs, and support for political
institutions. In general, public support for institutions is driven by a combination of
preferences over processes and outputs (Doherty, 2015b; Gangl, 2003; Jones &
McDermott, 2010; Ramirez, 2009, 2013; Skitka, Winquist, & Hutchinson, 2003; Tyler,
2001). One way that processes and outputs are connected is through the degree of
partisan conflict. The degree of partisan conflict on an issue can shape policy outputs, as
one side may emerge victorious with policy reflecting their views on the issue. But
partisan conflict can also shape process by, for example, setting the tone for
congressional debate on an issue and affecting legislators’ willingness to work out
5
compromises. Indeed, both members of Congress and the mass public see the degree of
partisanship as an important dimension on which to evaluate congressional performance.
Members have been known to equate partisan conflict with “bad” policymaking and
bipartisan cooperation with “good” policymaking (Manley, 1965). The polls outlined
earlier—as well as the consistent over-time relationship between partisan conflict and
institutional approval (e.g., Ramirez, 2009)—suggest that the public sees the degree of
partisanship as an important evaluative criterion as well.
The public’s resulting evaluations of Congress are important for several reasons.
They can affect the reelection chances of incumbents (Born, 1990; Jones & McDermott,
2010), discourage prospective candidates for running for office (Fowler & McClure,
1989), and impact the perceived legitimacy of the institution and its outputs (Gibson et
al., 2005; Hibbing & Theiss-More, 2001; Tyler, 1994). As noted by Durr et al. (1997, p.
177), “Without the support of the governed, the already difficult legislative process may
become even more so, and the policy that emerges may lack a sense of legitimacy.” For
all of these reasons, it is important to understand the connections between styles of
governing, policy outputs, and support for the institution.
Focusing on public preferences over partisanship and compromise, we agree that
citizens should prefer party conflict to compromise when it can be reasonably expected to
result in a policy victory for one’s own party (e.g., Harbridge et al. 2014). However, we
depart from existing work by emphasizing another possible result of party conflict:
legislative gridlock. Citizens expect their representatives to act as problem solvers and to
take action on pressing national problems (Adler & Wilkerson, 2013; Butler & Powell,
2014). Thus, while partisans clearly prefer to see their party’s proposals enacted, they
6
may recognize that party conflict raises the specter of gridlock, which is undesirable since
it prevents Congress from acting on pressing problems.
When presented with the outcome of party conflict, we expect citizens to act as
existing literature suggests and reward (punish) Congress when it takes action that
comports (conflicts) with their partisan objectives. We also expect citizens to reward
Congress for finding bipartisan solutions (i.e., for “compromising”), but less so than
when party conflict produces a “partisan win” for one’s own party. Contrary to the
existing wisdom, however, we expect citizens to reward Congress for taking action on a
pressing problem, even if the result is a “partisan win” for the opposing party. We
recognize that this expectation runs counter to the prevailing view of partisans as
preferring policies endorsed by their own party (Gerber & Huber, 2010; Lavine,
Johnston, & Steenbergen, 2012). However, when there is broad consensus about policy
goals (even as there is disagreement about the means to achieve those goals), citizens
may reward action on important problems, regardless of whether policy changes reflect
the liberal or conservative position. As Egan (2014) explains, people may have doublepeaked preferences – preferring policy change in either ideological direction over the
status quo – on issues where there is consensus over goals, where the problem is viewed
as serious, and where credible alternatives to the status quo are provided by both parties.
While the parties regularly offer competing solutions to problems that are viewed as
serious, the extent of agreement over end goals varies substantially. As a result, we
contend that on consensus issues (i.e., issues characterized by disagreement over means
but agreement over end goals), gridlock will be viewed more negatively than a win by the
opposing party, whereas on non-consensus issues, gridlock will be viewed more similarly
7
to a win by the opposing party. In sum, we expect to observe the following relationships
across legislative outcomes:
H1: Approval of Congress should be higher when one’s own party wins than
when the opposing party wins or when the parties reach a compromise.
H2a (Consensus Issue): On a consensus issue, approval of Congress should be
higher when the opposing party wins than when debate ends in legislative
gridlock.
H2b (Non-Consensus Issue): On a non-consensus issue, approval of Congress
should be no different when the opposing party wins and when debate ends in
legislative gridlock.
Types of Gridlock
Until very recently, the dominant explanation for party polarization and
legislative gridlock has been rooted in ideological differences between the two parties.
However, party conflict need not be based on ideological disagreements (Lee, 2009;
Noel, 2013). Strategic politicians may have incentives to engage in party conflict even on
issues that do not directly impinge upon ideological principles (Gilmour, 1995), such as
good government causes and procedural votes (see Lebo, McGlynn, & Koger, 2007; Lee,
2009).
A large body of research suggests that media coverage of politics regularly
includes references to both ideological disagreements and parties’ strategic goals (see
Aalberg, Stromback, & de Vreese, 2011 for review). The same is true for the media’s
coverage of legislative gridlock. For instance, a Washington Post opinion story about the
2013 government shutdown frames it in terms of “expansive and explosive” ideological
8
differences and “true believers” who failed to compromise (Samuelson, 2013). In
contrast, an opinion story in The Week argues that the “government shutdown is being
driven by confusion, arrogance, political opportunism…It is most definitely not being
driven by principle” (Brandus, 2013). As these examples illustrate, citizens are presented
with numerous explanations for legislative outcomes, and these varying narratives may
affect public opinion differently.2
As discussed, we expect citizens to disapprove of congressional performance
when debate ends in gridlock. However, we expect citizens’ responses to gridlock to vary
depending on whether gridlock is attributed to ideological differences (“ideological
gridlock”) or to party strategy (“partisan gridlock”). Some degree of party conflict is
inevitable given the nature of American political institutions (e.g., multiple veto points).
Moreover, responsible parties scholarship suggests that the public should reward parties
for differentiating themselves and presenting the public with clear choices (Ramirez,
2009, 683). In reality, however, citizens generally respond negatively to party conflict
(Ramirez, 2009, 2013) and take a skeptical view of the parties’ motivations in the policy
process (Hibbing & Theiss-Morse, 2002). Along these lines, we suspect that attributing
gridlock to expressly partisan goals—such as denying the opposing party a legislative
2
We do not mean to suggest that these sources of gridlock are mutually exclusive – that
is, it is certainly possible that ideological disagreements and strategic partisan
considerations can jointly lead to inaction (see Lee, 2009). This paper represents a first
step toward understanding whether these competing narratives affect public opinion
differently.
9
victory in the run-up to an election—will result in lower evaluations than when gridlock
is attributed to genuine ideological differences. Thus, we hypothesize the following:
H3: Approval of Congress should be higher when gridlock is attributed to
ideological differences than when it is attributed to partisan fighting.
Study Design
We tested our expectations with two survey experiments in which we manipulated
aspects of the legislative process and the outcome of partisan conflict. To understand how
reactions to party conflict, compromise, and gridlock vary across issues, the first study
focused on energy policy and the second focused on gun ownership. Importantly, these
two policies differ on the extent to which both parties agree over end goals. On energy
policy, both parties largely agree on the goal of energy independence and lower costs for
consumers (i.e., consensus issue), whereas on gun ownership, the two parties disagree
over whether gun ownership should be expanded or contracted (i.e., non-consensus
issue). On both issues, however, and in both policy descriptions presented to participants,
the parties offer different policy solutions to address the issue. That is, the parties
disagree on the means of addressing both issues. The consensus/non-consensus
distinction is reflected in the frequency of double-peaked preferences on these issues
(Egan, 2014), and is confirmed by a pre-test of the issue descriptions used in our
experiments.3 People are significantly more likely to perceive agreement between the two
3
For both issues, we pre-tested the policy descriptions provided to survey participants,
finding that people viewed both descriptions as ideologically balanced but viewed parties
10
parties over the end goals on energy policy than on gun ownership.
Although these issues fit the consensus/non-consensus distinction well, there may
be other issue-level characteristics that could also affect public preferences for partisan
conflict, and reactions to gridlock in particular. First, one might worry that the
distribution of public preferences on these issues would bias us toward finding greater
support for “do something” politics (i.e., legislative action rather than gridlock) on
consensus issues. That is, if partisans in the public are substantially more likely to take
their party’s position on gun ownership than on energy policy, it could predispose us to
find a greater preference for action by the opposing party over gridlock on energy than on
gun ownership. However, in our own survey of Amazon Mechanical Turk participants
(the same sample population as our studies), we find that, if anything, the partisan divide
is greater on energy policy than on gun ownership policy. On energy, there is a 61.5
percentage point divide between the parties, compared to a 58 percentage point divide on
gun ownership.4 Both issues exhibit a substantial partisan divide, and to the extent there
are differences across issues, they bias us against finding support for our hypothesis about
the nature of consensus issues and a preference for action on energy policy.
as agreeing on the end goals of energy policy while disagreeing on the goals of gun
ownership policy.
4
When asked which position on energy policy comes closest to their own view, 87
percent of Democrats and 74.5 percent of Republicans took their own party’s position.
On gun ownership, 79.5 percent of Democrats and 78.6 percent of Republicans took their
party’s position.
11
Second, the location of current policy, or status quo in the event of inaction, could
be relatively more favorable for one party than the other, which could, in turn, affect
preferences for action. In our tests of hypotheses 2a and 2b (preferences for action even
by the opposing party over gridlock), we provide robustness checks of the results,
showing the patterns hold within both parties, which suggest that the location of the
status quo is unlikely to be driving the observed results. As a further check, we also
restrict our analyses to participants whose ideology matches their partisan identification,
or who can recall the position taken by their party on this issue (see Online Appendix 3
for more details). These restrictions do not change the key patterns, providing further
evidence that the status quo is unlikely to be driving our results.
Participants in both studies were recruited via Amazon’s Mechanical Turk
(MTurk) in 2014.5 Participants in study 1 (energy) were recruited between February and
March, and participants in study 2 (guns) were recruited in mid-November. In total, 693
participants in study 1 and 665 participants in study 2 completed the survey. The
demographics of our two sets of study participants are presented in Appendix A. While
MTurk samples are not nationally representative (Berinsky, Huber, & Lenz, 2012;
Krupnikov & Levine, 2014), the impact of sample characteristics on generalizability
hinges on whether we confront homogenous or heterogeneous treatment effects
(Druckman & Kam, 2011). Of particular issue for studies of public opinion are
partisanship, age, race, and education, all of which can affect political attitudes.
5
Amazon’s Mechanical Turk (MTurk) is an online labor market increasingly used in
leading political science research (see Online Appendix 2 for more details).
12
Robustness checks of the main results that follow show little evidence of heterogeneous
treatment effects of gridlock across these variables (see Online Appendix 2), suggesting
that our findings are likely to generalize beyond this sample population.
In both studies, participants began the survey by answering standard demographic
and political questions. For the experimental portion of each study, after reading some
background information about either energy policy or gun ownership policy (see Online
Appendix 1 for full text), participants were randomly assigned to one of six conditions,
which varied the legislative approach and outcome of policymaking: Compromise,
Democratic Win, Republican Win, Gridlock (unattributed), Gridlock (Ideological), or
Gridlock (Partisan) (Table 1 displays the full text of all treatments).6 We recoded the
second and third treatment conditions to indicate which party won a legislative victory:
one’s “own party” or the “other party.”7 This re-coding procedure allows us to analyze all
6
We designed the compromise treatment to highlight the fact that although the parties
have different positions, they were able to reach an agreement. Our description of their
positions and the resulting compromise is purposefully vague, and could capture either a
“classic compromise” where both sides made sacrifices, or a “consensual compromise”
where the parties focused only on places of common ground (Gutmann & Thompson,
2012, 12). The remaining five conditions describe instances in which the two parties
engage in some form of non-cooperation.
7
Following previous research, independents who indicated that they were closer to one of
the two parties were treated as partisans (Keith et al., 1992). Given our interest in
comparing outcomes that favor one’s own party or the opposing party, pure independents
13
participants together (i.e., to pool Democrats and Republicans in the “own party win” and
“other party win” conditions). Randomization checks confirm that conditions were
balanced on relevant pre-treatment covariates (see Appendix B).
[Table 1 about here]
After reading about the legislative outcome, participants were asked, “Do you
approve or disapprove of how Congress is handling the issue of [energy/gun
ownership]?” with a standard 7-point response scale ranging from “Strongly Approve” to
“Strongly Disapprove.” We rescaled responses to range between 0 and 1, with 0
indicating “Strongly Disapprove” and 1 indicating “Strongly Approve.” This dependent
variable captures issue-specific evaluations of Congress but also has important
consequences for broader evaluations of Congress. In particular, approval of how
Congress is handling each policy (energy or gun ownership) mediates the relationship
between our treatments and overall confidence in Congress. These patterns point to the
broader connections between styles of governance, policy outputs, and evaluations of the
institution. We return to these results in the following section.
A final point worth considering before presenting our results concerns the internal
and external validity of our experiments. Research designs high in internal validity (e.g.,
experiments) provide a crucial first step in establishing generalizable causal relationships.
As explained by McDermott (2011, p. 28), internal validity does not require that
experiments mimic the real world as long as subjects experience the relevant forces that
are excluded from the primary analyses but included in subsequent analyses that consider
the framing of gridlock since they are an important component of the electorate.
14
investigators want to elicit. In this article, we investigate how people respond to partisan
conflict when it is described as resulting in different outcomes, and when gridlock is
framed as occurring for different reasons. Both descriptions are common aspects of
media coverage, a point to which we return in the concluding section. By contrast,
external validity is best achieved by replicating studies across populations, issues, and
time periods, and by using a variety of methods (Druckman & Kam, 2011, p. 43;
McDermott, 2011, p. 34). Nonetheless, we took several steps to boost the external
validity of our experiments, such as considering different policy issues, connecting issue
positions to parties, and analyzing both issue-specific and overall Congressional
evaluations. In the end, our studies are well-positioned to uncover causal relationships,
while the generalizability of the broader findings is best addressed by examining these
questions in other settings and through alternative research designs.
Results
Policy Outcomes and Congressional Approval
To assess how citizens respond to compromise, party conflict, and gridlock, our
analysis proceeds in three steps. First, we consider evaluations in the four main
conditions: compromise, own party win, other party win, and gridlock. Given the
importance of partisan victories, these analyses focus on partisans in the public (including
independent “leaners”) and exclude pure independents. Second, we consider whether the
framing of gridlock as ideological or partisan alters evaluations. Here, we bring pure
independents back into the analysis. For steps one and two, we consider differences
between consensus and non-consensus issues. Third, we examine the mediational
relationship between our treatments, policy-specific approval, and overall confidence in
15
Congress.
Comparing approval of how Congress is handling policymaking across our four
main conditions, we see that approval hinges on both the process and outcome of
policymaking. Consistent with our first hypothesis, the rank ordering of approval across
the four possible policymaking approaches and outcomes on energy policy shows the
greatest support for Congress when the outcome is “own party win,” followed by
“compromise,” “other party wins,” and lastly, “gridlock.” These patterns are shown
graphically by the mean approval in each condition of study 1 (Figure 1, left-hand panel).
Not surprisingly, a win by one’s own party is preferred over all other outcomes, and
compromise is preferred over a win for the other party or gridlock. The substantive effect
of these differences is meaningful. Moving from a win for one’s own party to
compromise reduces approval by 10 percentage points (p<0.01).8 When the opposing
party wins, approval drops by 23 percentage points relative to one’s own party winning
(p<0.001) and by 13 percentage points relative to compromise (p<0.001).
Perhaps more surprising, but consistent with our expectation that the public values
policy action on consensus issues, people prefer a win by the opposing party over
gridlock. Compared to a win by the opposing party, approval of how Congress is handing
energy policy drops by 15 percentage points following gridlock (p<0.001). That is,
approval is lower when there is legislative inaction than when the opposing party secures
a policy victory. This effect holds within participants of each party as well, a pattern that
would not be expected if people did not have a preference for legislation action and
8
All reported p-values come from two-tailed tests.
16
simply responded to whether they viewed the status quo as favorable for their party (see
Online Appendix 3). Interestingly, this preference for legislative action – regardless of
the partisan direction – is apparent among both weak and strong partisans. Weak
partisans prefer to see policy change in the direction of the opposing party over gridlock
(difference=17 percentage points, p<0.001), as do strong partisans (difference=11
percentage points, p=0.09).9
[Figure 1 about here]
Shifting from a consensus issue in study 1 to a non-consensus issue in study 2
changes the results in hypothesized ways. Whereas the general dislike of legislative
inaction persists on gun ownership policy, the relative downside of gridlock to a win by
the opposing party declines (see Figure 1, right-hand panel). As in study 1, evaluations
are highest when one’s own party wins, followed by compromise, although the 4.4
percentage point difference is not significant (p=0.30). Likewise, evaluations are 21
percentage points higher when one’s own party wins compared to when the opposing
party wins (p<0.001), 17 percentage points higher when there is compromise than when
the opposing party wins (p<0.001), and 14 percentage points higher when there is
compromise than when there is legislative inaction (p<0.001). In contrast to study 1,
however, evaluations are statistically indistinguishable when the opposing party wins and
9
Although our focus here is on how individual-level factors affect the key pattern
observed above, other evidence suggests that partisans in our sample behave in a manner
consistent with theoretical expectations. For example, strong partisans significantly prefer
a win by their own party over compromise, while weak partisans view the two similarly.
17
when there is legislative gridlock (p=0.54). When partisan conflict results in gridlock,
approval is 2.6 percentage points higher than when the opposing party secures a victory.
This pattern is similar among Democrats and Republicans, although limited power in
analyses of Republican participants does not allow us to rule out the possibility that
Republicans actually prefer gridlock over a victory for the opposing party since the status
quo is more favorable to Republicans than to Democrats on this issue (see Online
Appendix 3). This pattern is also similar among both weak and strong partisans; neither
group favors legislative action by the opposing party over gridlock. As expected, the
preference for legislative action over gridlock hinges on issue area, not simply
partisanship.
Consistent with our expectation that policy action is valued and gridlock is
abhorred when both parties agree on the end goals, but that inaction is more preferable
when the parties disagree on end goals, study 1 yields higher evaluations when the
opposing party wins than when there is gridlock, while study 2 does not. The significance
of this pattern is confirmed in a test of the difference-in-differences between the two: the
15 percentage point difference between the other party winning and gridlock in study 1 is
significantly different from the -2.6 percentage point difference in study 2 (p<0.001).
Consistent with our expectations, these patterns point to similarities in the relationship
between own party victories, compromises, and opposing party victories across these two
issues, but key differences in the relationship between opposing party victories and
gridlock.
We now turn to examining whether different frames of gridlock alter evaluations
18
of Congress.10 Here, we find support for our hypothesis in study 1 but not in study 2. As
shown in the left-hand panel of Figure 2, on energy policy, people prefer gridlock that is
attributed to ideological differences over gridlock that is attributed to partisan fighting
(difference=6.6 percentage points, p=0.04). That is, approval is 6.6 percentage points
lower when gridlock is attributed to the parties refusing to grant the opposing side a
legislative victory relative to when it is attributed to the two sides having conflicting
principles. Moreover, on energy policy, people prefer gridlock that is attributed to
ideological differences over the generic form of gridlock that does not explain its cause
(difference=5.2 percentage points, p=0.09).
Which types of individuals differentiate between ideological and partisan gridlock
and adjust their evaluations accordingly? Evidence from study 1 suggests a potential role
for strength of partisanship. Among strong partisans, Congress is rated equivalently under
10
Because we are interested in the effect of the treatments on the treated, we drop
participants in these conditions who failed a series of manipulation checks at the very end
of the survey designed to capture whether participants were paying attention and
understood the legislative outcome and the reason behind it. While the preference for
action over gridlock in study 1 holds regardless of participant attentiveness, preferences
for ideological gridlock over partisan gridlock hold only among attentive participants
who recognized the distinction between these two frames. See Online Appendix 4 for
more details and, following Berinsky, Margolis, & Sances (2014), treatment effects
across various thresholds of attentiveness and demographic predictors of passing the
manipulation checks.
19
ideological and partisan gridlock (difference=2.7 percentage points, p=0.70). In contrast,
weak partisans are less tolerant of gridlock that is attributed to partisan fighting, as they
rate Congress 6.3 percentage points lower under partisan as opposed to ideological
gridlock, a difference that approaches significance (p=0.13).
However, in study 2, people react similarly to ideological and partisan gridlock
(difference=0.9 percentage points, p=0.81). This insignificant relationship holds among
both strong and weak partisans, suggesting that strength of partisanship does not affect
responses to the two frames of gridlock on this issue. Moreover, the difference-indifference between these treatments across study 1 (consensus) and study 2 (nonconsensus) is significant as well (difference=5.7 percentage points, p=0.03). These results
suggest that issue-based differences affect not only whether gridlock is viewed more
poorly than a win for the opposing party, but also whether people respond to different
framings of gridlock. On a non-consensus issue, people view the two forms equivalently,
suggesting that either the commitment to ideological principles does not boost
evaluations or efforts to score political points does not harm evaluations; both patterns
are possible when people view the parties as having opposing goals, and where gridlock
is no worse than a win for the other party. Although the data do not allow us to tease out
these differences, it may be that people expect partisan fighting on non-consensus issues
(where the parties are unlikely to agree even sans strategic partisanship), leading
evaluations to be similar across frames of gridlock on gun ownership but different on
energy.
[Figure 2 about here]
Combined, these patterns suggest that while partisan conflict resulting in a win for
20
one’s own party boosts approval relative to compromise, conflict resulting in gridlock
significantly damages approval. Moreover, gridlock that is framed as strategic rather than
ideological has the most corrosive impact on approval, at least on some issues. People do
not simply have preferences for congressional parties to engage in bipartisanship or
partisanship; their preferences are heavily dependent on the outcome of partisan conflict
and the type of issue at hand. Policy outputs and styles of governance are both important
for understanding evaluations. Thus, party leaders should be wary of pursuing a partisan
agenda if the risk of gridlock is high, particularly on consensus issues. If confronted with
institutional conditions that make gridlock likely, focusing attention on policy
compromises may be more likely to garner public support.
Although these studies utilized a convenience sample (MTurk), the observed
patterns are likely to generalize beyond this population. Robustness checks exploring the
possibility of heterogeneous treatment effects by party, age, race, and education yield few
significant interaction terms (see Online Appendix 2). Across the two primary
relationships presented above (i.e., preference for action by the opposing party over
gridlock, and preference for ideological over partisan gridlock), two studies, and four
variables, we find only two significant interactions. In both studies, Republicans are
significantly more supportive of partisan gridlock (relative to ideological gridlock) than
Democrats and Independents. Given that our sample over-represents Democrats, this
significant interaction suggests that the preference for ideological over partisan gridlock
may be limited to other Democrats and Independents. In contrast, we found no evidence
of heterogeneous treatment effects by party, age, race, or education on the preference for
action by the opposing party over gridlock, or by age, race, or education on the
21
preference for ideological over partisan gridlock. Combined, these patterns suggest a
highly homogenous effect of gridlock relative to the other side winning, and a potentially
more heterogeneous effect of the framing of gridlock.
Partisan Conflict, Issue-Specific Approval, and Confidence in Congress
To this point, we have considered the effects of our treatments on issue-specific
approval of Congress. We now briefly consider the implications of these effects for
broader evaluations of Congress. In particular, we focus on the extent to which issuespecific approval mediates the effect of our treatments on overall confidence in
Congress.11 Put differently, we consider whether our treatments affect issue-specific
approval, which then, in turn, affects overall confidence in Congress. We turn to causal
mediation analysis to disentangle these relationships (Imai, Keele, Tingley, &
Yamamoto, 2011). Causal mediation analysis entails decomposing the overall effect of a
treatment (e.g., a victory for one’s own party) on an outcome (e.g., confidence in
Congress) into indirect effects, which represent the mechanism of interest (e.g., issuespecific approval), and direct effects, which represent all other mechanisms (Imai et al.,
2011, p. 768). More generally, this approach allows us to consider the relationship
between legislative strategies and outcomes, issue-specific approval of Congress, and
overall evaluations of the institution.
11
As a second dependent variable, we asked participants how much confidence they had
in the U.S. Congress. Response options ranged from “none” to “a great deal” on a fivepoint scale. As with other measures, responses were recoded from 0-1, with one
indicating the highest level of confidence.
22
For each study, we focus on the first four treatments, again using the sub-sample
of partisans and independent “leaners.” We compare the effects of each treatment relative
to the compromise condition, and then also compare the effects of gridlock relative to the
opposing party wins condition. In each case, we consider confidence in Congress as the
dependent variable and issue-specific approval as the mediator. The results provide
strong evidence that issue-specific approval mediates the relationship between our
treatments and overall confidence in Congress, pointing to the importance of these
relationships for understanding broader evaluations of the institution. Interestingly,
among the possible outcomes of party conflict, only gridlock exerts a significant direct
effect on confidence in Congress in addition to the mediated effect.12
For instance, in study 1 (energy), partisan conflict resulting in a win for one’s own
party increases confidence in Congress by 2.1 percentage points (over compromise)
(p=0.39), reflecting a significant mediated effect of 4.8 percentage points (p<0.001) and
an insignificant direct effect of -2.6 percentage points (p=0.23). Similarly, a win for the
opposing party has a significant negative effect on confidence in Congress relative to
compromise (-8.6 percentage points, p<0.001), which also reflects a significant mediated
effect (-7.9 percent pages, p<0.001) and an insignificant direct effect (-0.007 percentage
points, p=0.76). The observed preference for legislative action over gridlock in study 1 is
also seen in aggregate congressional approval. The total effect of -6.6 percentage points
(p=0.01) reflects a significant indirect effect of gridlock (over the other party winning) (-
12
Complete results of the mediation analysis and accompanying sensitivity analyses are
presented in Online Appendix 5.
23
5.5 percentage points, p<0.001) and an insignificant direct effect (-1.1 percentage point,
p=0.62). In contrast, partisan conflict resulting in gridlock has both a significant mediated
effect (relative to compromise) (-10.9 percentage points, p<0.001) and a significant direct
effect (-4.5 percentage points, p=0.08) on confidence in Congress.
These patterns – which are similar in studies 1 and 2 – suggest two important
points. First, issue-specific approval is consequential for broader evaluations of Congress.
Second, gridlock is distinct from other consequences of partisan conflict (i.e., partisan
wins and losses) insofar as it has both direct and indirect effects on confidence in
Congress. Although more research is needed to disentangle why gridlock is distinct in
this regard, the results are suggestive that gridlock can decrease confidence in Congress,
even absent people considering how Congress is addressing a particular policy.
Discussion
Past research on party conflict and public opinion has yielded contradictory
results: the public consistently disapproves of congressional performance during periods
of party conflict, yet often rewards legislators for engaging in partisanship. The present
study speaks to this puzzle by emphasizing the implications of party conflict for
legislative outputs. The distinction between partisan victories and legislative gridlock is
important to understanding this puzzle, as partisan conflict often results in inaction
(Binder, 2003). However, previous research has focused on party conflict in general (e.g.,
Ramirez, 2009), or on conflict resulting in partisan victories (e.g., Harbridge et al., 2014).
We extended this line of inquiry by examining whether the public’s response depends on
whether conflict prevents legislative action. Our research made three primary
contributions to the literature on party conflict in Congress and public opinion.
24
First, we moved beyond past work by varying both legislative strategy (i.e.,
cooperation or conflict) and its effects on policy outcomes. While our participants acted
in line with their partisan allegiances, favoring wins for their own party, we found
convincing evidence that citizens respond favorably when Congress acts on a pressing
problem—even if that action is contradictory to one’s partisan goals. For instance, we
found that on a consensus issue where the parties agree on the ultimate end goal, citizens
evaluate Congress more favorably when partisan conflict results in a victory for the
opposing party than when conflict results in inaction. In contrast to scholarly accounts
emphasizing the dominating influence of partisanship, it appears that citizens – strong
and weak partisans alike – are not only open to compromise but are even open to policy
proposals from the other party (when the alternative is gridlock). However, when
confronted with a non-consensus issue where the parties disagree over the end goal,
gridlock and a victory for the opposing party are viewed much more similarly.
Second, we unpacked “gridlock” by testing the influence of alternate narratives—
ideological differences versus party strategy—on public opinion. This distinction was
motivated by the increasingly fragmented media environment, in which citizens are often
presented with competing explanations for party conflict and polarization. Consistent
with our predictions, we found that on energy policy citizens are more accepting of
gridlock when it is attributed to sincere ideological disagreements between the two
parties rather than party strategy. This pattern was driven by independents and weak
partisans, as strong partisans appear less concerned by partisan fighting. Although the
evidence was weaker on gun ownership policy – where the non-consensus nature of the
policy may lead people to expect some degree of partisan strategizing – the results
25
suggest an important role for elites in framing the outcome of policy debates. Levendusky
(2013, p. 36) makes a similar point in discussing the importance of media framing for
citizens’ understanding of election outcomes. In his discussion of partisan media, he
argues that elites can shape important post-election attitudes, such as the perceived
legitimacy of a new regime. In the context of the present study, this insight suggests that
the public’s reaction to prominent cases of legislative gridlock depends in part on which
explanation for gridlock prevails in the media.
Third, we emphasized that reactions to legislative strategies and outcomes are
contingent on issue area. Public evaluations of Congress fall much more considerably
when partisan conflict results in legislative gridlock on a consensus issue than on a nonconsensus issue. On the former (e.g., energy policy), people prefer a win by the opposing
party over legislative gridlock. On the latter (e.g., gun ownership policy), people evaluate
the two outcomes similarly. Thus, the potential downsides to party conflict – especially
when it results in gridlock rather than a victory for one party – are larger on some issues
relative to others.
Our results are all the more surprising given that we explicitly connected policy
outcomes to wins for a particular party. In actual campaigns and policy debates,
arguments and outcomes are typically clearly linked to political parties (e.g., Slothuus &
de Vreese, 2010). However, while citizens sometimes endorse a range of policy solutions
to pressing national problems (Egan, 2014), it was unclear from existing work whether
this finding holds when proposals are clearly linked to parties. To address this gap, and to
boost the external validity of our own experiments, we clearly linked policy outcomes to
a particular party, and found that citizens are in fact open to proposals from the opposing
26
party (on consensus issues).
Moreover, our studies reflect how policymaking, including partisan conflict and
gridlock, is often discussed in the media, bolstering the experimental realism of our
treatments and the external validity of our findings. Political communication research
suggests that the media often eschew detailed coverage of policies in favor of “game
framing,” which highlights the “winners and losers” of policymaking (Aalberg et al.,
2011). As Lawrence (2000, p. 98) points out, game framing is especially likely when
policymakers engage in legislative conflict.
While some issues produce disagreement over who emerged victorious, on many
issues there is widespread agreement about which party secured a legislative victory. For
instances, the Affordable Care Act was described as “a historic victory” for Democrats
(Murray & Montgomery, 2010), while “fast track” authority on trade was described as a
Republican victory (Babington & Espo, 2015). Other issues, including cap-and-trade
policies for energy (Walsh, 2010) and reforms to background checks on gun purchases
(Madison, 2013) were discussed as clear instances of gridlock. Moreover, many news
stories on both energy and gun ownership provided an explanation for gridlock similar to
the ideological and partisan explanations found in our treatments. Journalists spoke of
“myriad interests and views” as a recipe for gridlock on energy reform (Koss, 2008), and
a “false sense of ideological purity” driving gridlock on gun ownership (Katz, 2013).
They also referenced “partisan gridlock” resulting from politicians trying to “ram through
sweeping measures” on carbon emissions (Broder & Krauss, 2010), and “the partisan
gridlock that has paralyzed Washington” (Nagourney, 2014). Likewise, the failure of
background check legislation was attributed to “hyper-partisanship and its [Congress’]
27
own self-imposed rules” (Lesley, 2013). As these examples illustrate, our treatments
capture many of the same elements of how gridlock is framed in actual media coverage.
To be sure, this is not to suggest that our experiments capture all the complexities
of the contemporary media environment. For instance, our design does not incorporate
competing explanations for gridlock, nor does it allow participants to self-select into
certain types of media coverage. Competition and choice can moderate media effects on
political attitudes (Busby, Flynn, & Druckman, Forthcoming; Druckman & Lupia,
Forthcoming), and future research should incorporate these realities into studies of
partisan conflict and public opinion.
Our scholarly contributions come with important lessons for practitioners of
legislative politics. Recent years have seen historically high levels of party conflict and
historically low levels of public approval of Congress. Our work suggests an important
corollary to the party conflict-approval link: the role of legislative outcomes. Legislative
gridlock, such as the 2013 government shutdown, can damage congressional approval
(Newport, 2013). Thus, when crafting legislative strategies, electorally minded leaders
need to consider the likely outcome of party conflict (i.e., partisan win versus gridlock)
and the type of issue at hand. To the extent that members of the majority party are
affected electorally by institutional approval (Jones, 2010), majority party leaders have
greater incentives to ensure the passage of legislation, including bipartisan compromises,
when the risk of gridlock is significant. Ensuring legislative action—from either party—
is all the more important on consensus issues. When faced with divided government,
proposals that can garner bipartisan support may be required on these issues. In contrast,
if parties focus on non-consensus issues – such as gun ownership, abortion, and the place
28
of religion in the public sphere – gridlock may be less costly to leaders, and they may be
able to safely pursue legislation that achieves partisan “position taking.” Thus, better
understanding the relationships between party strategies, legislative outcomes, and public
opinion across issues is important not only for scholarly theories of parties and
representation, but for party leaders as well.
29
References
Aalberg, T., Stromback, J., & de Vreese, C. H. (2011). The framing of politics as strategy
and game: A review of concepts, operationalizations, and key findings.
Journalism, 13(2), 162-178.
Adler, E. S., & Wilkerson, J. D. (2013). Congress and the politics of problem solving.
New York: Cambridge University Press.
Babington, C., & Espo, D. (2015, June 24). Republican-led Congress hand Obama major
win on trade. Associated Press. Retrieved from http://news.yahoo.com/senatepoised-hand-obama-big-victory-negotiating-trade-071058503--finance.html
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for
experimental research: Amazon.com's Mechanical Turk. Political Analysis, 20(3),
351-368.
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from
the workers? Making sure respondents pay attention on self-administered surveys.
American Journal of Political Science, 58(3), 739-753.
Binder, S. (2003). Stalemate: causes and consequences of legislative gridlock.
Washington D.C.: Brookings Institution Press.
Binder, S. (2014). Polarized we govern? Brookings Institution. Retrieved from
http://www.brookings.edu/~/media/research/files/papers/2014/05/27-polarizedwe-govern-binder/brookingscepm_polarized_figreplacedtextrevtablerev.pdf
Born, R. (1990). The shared fortunes of Congress and congressmen: Members may run
from Congress, but they can't hide. Journal of Politics, 52(4), 1223-1241.
Brandus, P. (2013, September 30). Who will 'win' in the government shutdown?
30
Retrieved from http://theweek.com/article/index/250355/who-will-win-thegovernment-shutdown#axzz33armcb4u
Broder, J. M., & Krauss, C. (2010, January 27). Advocates of climate bill scale down
their goals. The New York Times. Retrieved from
http://www.nytimes.com/2010/01/27/science/earth/27climate.html
Brooks, D. J., & Geer, J. G. (2007). Beyond negativity: The effects of incivility on the
electorate. American Journal of Political Science, 51(1), 1-16.
Burden, B. C. (2011). Polarization, obstruction, and governing in the Senate. The Forum,
9(4). Retrieved from http://www.degruyter.com/view/j/for.2011.9.issue-4/15408884.1480/1540-8884.1480.xml
Busby, E., Flynn, D. J., & Druckman, J. N. (Forthcoming). Putting framing effects in
their place: When frames (may) matter. In P. D'Angelo (Ed.), Doing news framing
analysis II. New York: Routledge.
Butler, D. M., & Powell, E. N. (2014). Understanding the party brand: Experimental
evidence on the role of valence. Journal of Politics, 76(02), 492-505.
Canes-Wrone, B., Brady, D. W., & Cogan, J. F. (2002). Out of step, out of office:
Electoral accountability and House members' voting. American Political Science
Review, 96(1), 127-140.
Cappella, J. N., & Jamieson, K. H. (1997). Spirals of cynicism: The press and the public
good. New York: Oxford University Press.
Carson, J. L., Koger, G., Lebo, M. J., & Young, E. (2010). The electoral costs of party
loyalty in Congress. American Journal of Political Science, 54(3), 598-616.
CBS News. (2009, July 13). "President Obama, the economy, and foreign affairs." CBS
31
News Poll. Retrieved from
http://www.cbsnews.com/htdocs/pdf/poll_Obama_071309.pdf
Doherty, D. (2015a). How policy and procedure shape citizens' evaluations of senators.
Legislative Studies Quarterly, 40(2), 241-272.
Doherty, D. (2015b). Perceived motives in the political arena. American Politics
Research, 43(3), 363-393.
Druckman, J. N., & Kam, C. D. (2011). Students as experimental particpants: A defense
of the 'narrow data base.' In J. N. Druckman, D. P. Green, J. H. Kuklinski, & A.
Lupia (Eds.), Cambridge handbook of experimental political ccience (pp. 41-57).
New York: Cambridge University Press.
Druckman, J. N., & Lupia, A. (Forthcoming). Preference change in competitive political
environments. Annual Review of Political Science. Available at
http://faculty.wcas.northwestern.edu/~jnd260/pub/Druckman%20Lupia%20Prefer
ence%20Change.pdf.
Druckman, J. N., Peterson, E., & Slothuus, R. (2013). How elite partisan polarization
affects public opinion formation. American Political Science Review, 107(1), 5779.
Durr, R. H., Martin, A. D., & Wolbrecht, C. (1997). Explaining congressional approval.
American Journal of Political Science, 41(1), 175-207.
Egan, P. J. (2014). Double-peaked policy preferences and 'do something' politics. Journal
of Politics, 76(2), 333-349.
Fowler, L. L., & McClure, R. D. (1989). Political ambition: who decides to run for
Congress? New Haven: Yale University Press.
32
Fridkin, K. L., & Kenney, P. (2011). Variability in citizens' reactions to different types of
negative campaigns. American Journal of Political Science, 55(2), 307-325.
Gangl, A. (2003). Procedural justice theory and evaluations of the lawmaking process.
Political Behavior, 25(2), 119-149.
Gerber, A. S., & Huber, G. A. (2010). Partisanship, political control, and economic
assessments. American Journal of Political Science, 54(1), 153-173.
Gibson, J. L., Caldeira, G. A., & Spence, L. K. (2005). Why do people accept public
policies they oppose? Testing legitimacy theory with a survey-based experiment.
Political Research Quarterly, 58(2), 187-201.
Gilmour, J. B. (1995). Strategic disagreement. Pittsburgh: University of Pittsburgh Press.
Gutmann, A., & Thompson, D. (2012). The spirit of compromise: Why governing
demands it and campaigning undermines it. Princeton, NJ: Princeton University
Press.
Harbridge, L., & Malhotra, N. (2011). Electoral incentives and partisan conflict in
Congress: Evidence from survey experiments. American Journal of Political
Science, 55(3), 1-17.
Harbridge, L., Malhotra, N., & Harrison, B. F. (2014). Public preferences for
bipartisanship in the policymaking process. Legislative Studies Quarterly, 39(3),
327-355.
Hibbing, J. R., & Theiss-More, E. (Eds.). (2001). What is it About government that
Americans dislike? New York: Cambridge University Press.
Hibbing, J. R., & Theiss-Morse, E. (1995). Congress as public enemy: Public attitudes
toward American political institutions. New York: Cambridge University Press.
33
Hibbing, J. R., & Theiss-Morse, E. (2002). Stealth democracy. New York: Cambridge
University Press.
Imai, K., Keele, L., Tingley, D., & Yamamoto, T. (2011). Unpacking the black box of
causality: Learning about causal mechanisms from experimental and
observational studies. American Political Science Review, 104(4), 765-789.
Iyengar, S., Sood, G., & Lelkes, Y. (2012). Affect, not ideology: A social identity
perspective on polarization. Public Opinion Quarterly, 76(3), 405-431.
Jones, D. R. (2001). Party polarization and legislative gridlock. Political Research
Quarterly, 54(1), 125-141.
Jones, D. R. (2010). Partisan polarization and congressional accountability in House
elections. American Journal of Political Science, 54(2), 323-337.
Jones, D. R. (2013). Do major policy enactments affect public evaluations of Congress?
The case of health care reform. Legislative Studies Quarterly, 38(2), 185-204.
Jones, D. R., & McDermott, M. L. (2010). Americans, Congress, and democratic
responsiveness: public evaluations of Congress and electoral consequences. Ann
Arbor: University of Michigan Press.
Katz, C. (2013, April 17). Reaction roundup: Manchin-Toomey gun check amendment
fails in Senate. New York Daily News. Retrieved from
http://www.nydailynews.com/blogs/dailypolitics/reaction-roundup-manchintoomey-gun-check-amendment-fails-senate-blog-entry-1.1694324
Keith, B. E., Magleby, D. B., Nelson, C. J., Orr, E., Westlye, M. C., & Wolfinger, R. E.
(1992). The myth of the independent voter. Los Angeles: University of California
Press.
34
Koss, G. (2008, March 31). All sides gird for cap-and-trade fight. Roll Call.
Krehbiel, K. (1998). Pivotal politics. Chicago: University of Chicago Press.
Krupnikov, Y., & Levine, A. S. (2014). Cross-sample comparisons and external validity.
Journal of Experimental Political Science, 1(01), 59-80.
Lavine, H., Johnston, C., & Steenbergen, M. (2012). The ambivalent partisan. New York:
Oxford University Press.
Lawrence, R. G. (2000). Game-framing the issues: Tracking the strategy frame in public
policy news. Political Communication, 17(2), 93-114.
Lebo, M. J., McGlynn, A. J., & Koger, G. (2007). Strategic party government: Party
influence in Congress, 1789-2000. American Journal of Political Science, 51(3),
464-481.
Lee, F. E. (2009). Beyond ideology: Politics, principles, and partisanship in the U.S.
Senate. Chicago: University of Chicago Press.
Lesley, B. (2013, April 22). Gun safety, gridlock, and the demise of Congress. The
Huffington Post. Retrieved from http://www.huffingtonpost.com/brucelesley/gun-safety-gridlock-and-t_b_3133220.html
Levendusky, M. (2013). How partisan media polarize America. Chicago: University of
Chicago Press.
Madison, L. (2013, April 18). Background checks voted down; Senate gun bill in peril.
CBS News. Retrieved from http://www.cbsnews.com/news/background-checksvoted-down-senate-gun-bill-in-peril/
Manley, J. F. (1965). The House Committee on Ways and Means: Conflict management
in a congressional committee. American Political Science Review, 59(4), 927-939.
35
Mann, T. E., & Ornstein, N. J. (2006). The broken branch: How Congress is failing
America and how to get it back on track. New York: Oxford University Press.
Mann, T. E., & Ornstein, N. J. (2012). It's even worse than it looks: How the American
constitutional system collided with the politics of extremism. New York: Basic
Books.
McCarty, N., Poole, K. T., & Rosenthal, H. (2006). Polarized America: The dance of
ideology and unequal riches. Cambridge: Massachusetts Institute of Technology
Press.
McDermott, R. (2011). Internal and external validity. In J. N. Druckman, D. P. Green, J.
H. Kuklinski, & A. Lupia (Eds.), Cambridge handbook of experimental political
science (pp. 27-40). New York: Cambridge University Press.
Murray, S., & Montgomery, L. (2010, March 22). House passes health-care reform bill
without Republican votes. The Washington Post. Retrieved from
http://www.washingtonpost.com/wpdyn/content/article/2010/03/21/AR2010032100943.html
Mutz, D. C., & Reeves, B. (2005). The new videomalaise: Effects of televised incivility
on political trust. American Political Science Review, 99(1), 1-15.
Nagourney, A. (2014, January 15). After flurry of changes, some states ease up. New
York Times. Retrieved from http://www.nytimes.com/2014/01/15/us/politics/afterflurry-of-changes-some-states-ease-up.html
Newport, F. (2013, October 7). Congress' job approval falls to 11% amid gov't shutdown.
Retrieved from http://www.gallup.com/poll/165281/congress-job-approval-fallsamid-gov-shutdown.aspx
36
Noel, H. (2013). Political parties and political ideologies in America. New York:
Cambridge University Press.
Paris, C. (2015). Costs nothing and buys everything: Incentives for congressional civility.
Paper presented at the Annual Meeting of the Midwest Political Science
Association, Chicago.
Pew. (2012). Trends in American values: 1987-2012. Retrieved from http://www.peoplepress.org/files/legacy-pdf/06-04-12%20Values%20Release.pdf
Pew. (2015). Congressional approval. Retrieved from http://www.pewresearch.org/datatrend/political-attitudes/congressional-favorability/
Ramirez, M. D. (2009). The dynamics of partisan conflict on congressional approval.
American Journal of Political Science, 53(3), 681-694.
Ramirez, M. D. (2013). The policy origins of congressional approval. Journal of Politics,
75(1), 198-209.
Samuelson, R. J. (2013, October 6). Ideology is what has won in the shutdown debate.
Washington Post. Retrieved from
http://www.washingtonpost.com/opinions/robert-samuelson-the-shutdown-is-atriumph-of-ideology/2013/10/06/1bc17054-2d4c-11e3-97a3ff2758228523_story.html
Skitka, L., Winquist, J., & Hutchinson, S. (2003). Are outcome fairness and outcome
favorability distinguishable psychological constructs? A meta-analytic review.
Social Justice Research, 16(4), 309-341.
Slothuus, R., & de Vreese, C. H. (2010). Political parties, motivated reasoning, and issue
framing effects. Journal of Politics, 72(3), 630-645.
37
Theriault, S. M. (2008). Party polarization in Congress. New York: Cambridge
University Press.
Tyler, T. R. (1994). Governing amid diversity: The effect of fair decisionmaking
procedures on the legitimacy of government. Law & Society Review, 28(4), 809831.
Tyler, T. R. (2001). The psychology of public dissatisfaction with government. In J. R.
Hibbing & E. Theiss-More (Eds.), What is it about government that Americans
dislike? (pp. 227-242). New York: Cambridge University Press.
Walsh, B. (2010, July 22). Cap and trade is dead (really, truly, I'm not kidding). Who's to
blame? Time. Retrieved from http://science.time.com/2010/07/22/cap-and-tradeis-dead-really-truly-im-not-kidding-whos-to-blame/
38
Table 1: Treatment Conditions (same in Studies 1 and 2)
Condition
Wording
1. Compromise
Despite their differences, a compromise
version of the bill was agreed on by both
sides and passed.
2. Democrats Win
The final version of the bill that passed
favored the Democratic priorities.
3. Republicans Win
The final version of the bill that passed
favored the Republican priorities.
4. Gridlock (unattributed)
Despite discussion of various proposals,
the legislation died in Congress and no
bill was passed.
5. Gridlock (ideological)
Despite discussion of various proposals,
neither side was willing to sacrifice their
principles on the issue. Without this giveand-take, the legislation died in Congress
and no bill was passed.
6. Gridlock (partisan)
Despite discussion of various proposals,
neither side was willing to hand the other
a victory on this issue in the run up to the
next election. Without this give-and-take,
the legislation died in Congress and no
bill was passed.
39
Figure 1: Mean Approval of How Congress is Handling Policy (Study 1 Energy, Study 2
Gun Ownership)
Note: Brackets contain 90% confidence intervals. Analysis restricted to partisans.
40
Figure 2: Mean Approval of How Congress is Handling Policy (Study 1 Energy, Study 2
Gun Ownership)
Note: Brackets contain 90% confidence intervals. Analysis includes partisans and pure
independents.
41
Appendix A: Sample Demographics
Table A1: Sample Demographics
Study 1 (Energy)
Mean Age
33.9
Mean Ideology
3.40
% Democrat (including
61.0
leaners)
% White
79.5
% Male
60.5
% College Degree+
56.7
Study 2 (Gun Ownership)
33.9
3.30
51.0
77.8
58.9
54.6
42
Appendix B: Randomization Check
Table B1: Randomization Check of Treatments (Study 1)
Compromise
Democrats
Win
Republicans
Win
Gridlock
Ideological
Gridlock
Partisan
Gridlock
Gender
Female
Male
2(5) = 6.1, p=0.30
35.8%
64.2
37.4%
62.6
35.9%
64.1
37.4%
62.6
42.1%
57.9
48.7%
51.3
Race
Nonwhite
White
2(5) = 5.9, p=0.32
22.8
77.2
21.3
78.7
25.2
74.8
13.1
86.9
18.4
81.6
21.7
78.3
8.9
30.1
52.0
8.9
9.9
41.2
36.6
12.2
7.8
35.0
41.7
15.5
9.3
29.9
49.5
11.2
9.6
37.7
35.1
17.5
8.7
30.7
43.9
16.7
68.8
9.8
21.4
59.3
13.0
27.6
56.7
13.4
23.0
64.0
13.0
23.0
66.7
14.8
18.5
50.5
21.6
27.9
52.0
28.4
16.3
3.3
55.0
27.5
11.5
6.1
38.9
44.7
13.6
2.9
44.9
31.8
15.0
8.4
47.4
34.2
12.3
6.1
46.1
28.7
23.5
1.7
123
131
103
107
114
115
Education
HS or less
Some College
Bachelors
Graduate Degree
2(15) = 15.6,
p=0.41
Party Identification
Democrat
Pure Independent
Republican
2(10) = 14.5, p=
0.15
Age
18-29
30-44
45-59
60+
2(15) = 25.6,
p=0.042
N
43
Table B2: Randomization Check of Treatments (Study 2)
Compromise
Democrats
Win
Republicans
Win
Gridlock
Ideological
Gridlock
Partisan
Gridlock
Gender
Female
Male
2(5) = 6.4, p=0.27
48.1%
51.9
47.0%
53.0
37.0%
63.0
39.8%
60.2
38.4%
61.4
35.4%
64.6
Race
Nonwhite
White
2(5) = 8.5, p=0.13
25.0
75.0
21.6
78.4
23.0
77.0
14.2
85.8
20.6
79.4
30.0
70.0
11.9
33.0
46.8
8.3
8.6
36.2
38.8
16.4
10.0
32.0
48.0
10.0
8.0
46.0
34.5
11.5
17.6
32.0
39.2
11.2
5.0
30.0
50.0
15.0
54.6
26.9
18.5
52.6
24.1
23.3
58.0
24.0
18.0
44.6
32.1
23.2
45.6
30.4
24.0
52.0
28.6
19.4
42.2
34.9
18.3
4.6
47.9
35.9
11.1
5.1
44.0
33.0
12.0
11.0
45.1
37.2
11.5
6.2
51.6
29.4
12.7
6.3
49.0
37.0
12.0
2.0
109
117
100
113
126
100
Education
HS or less
Some College
Bachelors
Graduate Degree
2(15) = 24.8,
p=0.052
Party Identification
Democrat
Pure Independent
Republican
2(10) = 7.0, p= 0.72
Age
18-29
30-44
45-59
60+
2(15) = 13.6,
p=0.55
N
44
[End of Paper]
Supporting Materials for “How Partisan Conflict in Congress Affects Public
Opinion: Strategies, Outcomes, and Issue Differences”
Online Appendix 1: Full Text of Study Treatments and Dependent Variables
Study 1
Background Information on Energy Policy (all participants)
Energy policies have important implications for many economic, environmental, and
social issues. In a recent session of Congress, legislation was discussed that sought to
address many aspects of energy policy by developing a more comprehensive national
energy plan. While the proposals from both the Democratic and Republican parties aimed
to create jobs in the energy sector, cut costs for consumers, and make America more
energy independent, the two parties differed on some aspects of the legislation and in the
priority they placed on the various components of a national energy policy.
Republicans favor continued support of existing energy sources, notably oil and gas.
With respect to renewable energies, Republicans favor letting market-based forces
determine the viability of new energy technology. In contrast, Democrats favor greater
restrictions on oil and gas production, particularly where the environmental risks are
uncertain, and greater government incentives to promote the development of renewable
energy technologies.
[TREATMENT – See Table 1]
Study 2
Background Information on Gun Policy (all participants)
Gun ownership is an important policy area with wide-reaching consequences for our
society. In a recent session of Congress, legislation was discussed that sought to address
many aspects of gun ownership. Proposals from the Democratic and Republican parties
differed in many respects.
Republicans favor preserving gun rights so that law-abiding citizens can protect
themselves against criminals and participate in sporting activities requiring firearms (e.g.,
hunting). In contrast, Democrats favor greater gun regulation in order to keep weapons
out of the hands of criminals and others who are incapable of using firearms responsibly.
[TREATMENT – See Table 1]
45
Dependent Variable Wording
Based on what you just read, how much confidence do you have in the U.S. Congress?
__ a great deal
__ a lot
__ a moderate amount
__ a little
__ none
Do you approve or disapprove of how Congress is handling [energy/gun ownership]
policy?
__ Strongly Approve
__ Somewhat Approve
__ Slightly Approve
__ Neither Approve Nor Disapprove
__ Slightly Disapprove
__ Somewhat Disapprove
__ Strongly Disapprove
46
Online Appendix 2: Amazon Mechanical Turk Participants and Testing for
Heterogeneous Treatment Effects
Survey participants were recruited using Amazon’s Mechanical Turk (MTurk), an
online labor market increasingly used in leading political science research (e.g., Ahler,
2014; Arceneaux, 2012; Bishin, Hayes, Incantalupo, & Smith, Forthcoming; Carnes &
Sadin, 2015; Christenson & Glick, 2015; Dowling & Wichowsky, 2015; Grose, Malhotra,
& Van Houweling, 2015; Healy & Lenz, 2013; Hersh & Schaffner, 2013; Ryan, 2014;
Testa, Hibbing, & Ritchie, 2014; Tomz & Weeks, 2013). Berinsky et al. (2012) report
that MTurk samples are more representative of the national population than frequently
used convenience samples (e.g., students). They also replicate several canonical
psychological experiments using MTurk samples. Moreover, Mullinix et al. (2014)
replicate studies fielded by Knowledge Network and find that even with differences in
demographics between the two, key results from 15 studies were largely similar using
MTurk samples (also see Paolacci, Chandler, & Ipeirotis, 2010).
Compared to national samples (e.g., American National Election Study,
Knowledge Networks, etc.), MTurk participants tend to over-represent the young,
students and those with college degrees, liberals, and self-identified Democrats; and to
under-represent minorities, and those who own their own home (Huff & Tingley,
Forthcoming; Krupnikov & Levine, 2014; Mullinix et al., 2014). Similar to other MTurk
studies, Democrats significantly outnumber Republicans in our samples. However, the
key findings within and between the two studies are evident even among just the
Democratic participants (see Table O1).
MTurk participants tend to produce as high quality data as other internet samples
like Knowledge Networks (Mullinix et al., 2014). Moreover, Clifford and Jerit (2014)
47
find that several common indicators of data quality—including correct responses to
manipulation and attention checks, and the reliability of multi-item scales—compare
favorably across laboratory and MTurk samples.
Important for our study in particular, past research has found that framing effects
operate similarly across nationally representative and MTurk samples, especially when
important moderators are taken into account (Berinsky et al., 2012; Leeper & Mullinix,
2014). More generally, most differences on political issues among MTurk participants
can be explained by these differences in demographics and partisanship. As a result, it is
important to consider the degree of heterogeneity in treatment effects. We focus in
particular on the potential moderating role of partisanship, age (as either a continuous
measure or an indicator for under 30), race, and education (indicator for college degree or
greater) on the two main effects reported in the paper. Table O2 presents a series of OLS
regression models in which we interact the treatment indicator for gridlock (capturing the
effect of gridlock relative to an opposing party win) with partisanship, age, race, and
education. Table O3 presents similar models for the effect of partisan gridlock (relative to
ideological gridlock). As can be seen in the tables, we find little evidence of
heterogeneous treatment effects. We uncover no significant interactions in Table O2,
suggesting that the effect of gridlock (relative to an opposing party win) is highly
homogeneous. Turning to the effect of partisan gridlock (relative to ideological gridlock)
in Table O3, only the interaction with Republican party identification yields a significant
result. Thus, in total, we find little evidence that the treatment effects we report are
moderated by important variables on which MTurk is known to differ from national
samples.
48
Table O1: Difference in Means among Democratic Participants
Study 1 (Energy)
Study 2 (Gun Ownership)
Compromise Own Win 0.48, 0.61 Compromise Own Win
0.48, 0.48
p=0.002
p=0.99
Compromise Other Win 0.48, 0.28 Compromise Other Win 0.48, 0.27
p<0.001
p<0.001
Compromise Gridlock
0.48, 0.18 Compromise Gridlock
0.48, 0.28
p<0.001
p<0.001
Other Party
Gridlock
0.28, 0.18 Other Party
Gridlock
0.27, 0.28
Win
p=0.006
Win
p=0.89
Ideological
Partisan
0.26, 0.17 Ideological
Partisan
0.30, 0.21
Gridlock
Gridlock
p=0.04
Gridlock
Gridlock
p=0.12
Cells contain: (mean in column 1, mean in column 2), with two-sided p-value on
difference below. Comparisons use the same subsets of participants as in the primary
analyses.
49
Table O2: Effect of Gridlock Relative to the Other Party Winning
Intercept
Gridlock
Republican
Gridlock x Republican
Age
(1)
0.282***
(0.0287)
-0.105**
(0.0392)
0.179***
(0.0465)
-0.0807
(0.0696)
—
(2)
0.423***
(0.0699)
-0.105
(0.0952)
—
Study 1
(3)
0.33***
(0.0299)
-0.16***
(0.0433)
—
(4)
0.349***
(0.0282)
-0.152***
(0.0381)
—
(5)
0.392***
(0.0366)
-0.182***
(0.0525)
—
—
—
—
—
—
—
—
(6)
0.27***
(0.0347)
0.00655
(0.051)
0.0694
(0.0615)
0.0514
(0.0887)
—
—
—
—
—
—
—
—
Gridlock x Age
—
Under 30
—
-0.00212
(0.00191)
-0.00107
(0.00253)
—
Gridlock x Under 30
—
—
Non-White
—
—
0.052
(0.0484)
0.0257
(0.0682)
—
Gridlock x Non-White
—
—
—
College Degree
—
—
—
0.00336
(0.0522)
0.0417
(0.0891)
—
Gridlock x College Degree
—
—
—
—
176
0.187
0.172
176
0.124
0.109
176
0.119
0.103
176
0.101
0.0857
N
R2
Adjusted R2
(7)
0.516***
(0.0873)
-0.0503
(0.125)
—
Study 2
(8)
0.236***
(0.0384)
0.029
(0.0542)
—
(9)
0.26***
(0.034)
0.0566
(0.0468)
—
(10)
0.305***
(0.0449)
0.0628
(0.0618)
—
—
—
—
—
—
—
—
—
—
—
—
-0.00662**
(0.00244)
0.00237
(0.00343)
—
—
—
—
—
—
—
—
—
—
—
0.123*
(0.0566)
0.012
(0.0833)
—
—
—
—
—
—
-0.0714
(0.0479)
0.0606
(0.0683)
176
0.111
0.0955
—
—
—
0.115^
(0.0639)
-0.0983
(0.118)
—
—
—
—
—
161
0.0323
0.0138
161
0.0646
0.0467
161
0.0603
0.0423
161
0.0228
0.00418
—
-0.0214
(0.0585)
-0.0803
(0.0845)
161
0.0206
0.00187
Note: OLS regression of approval of how Congress is handling energy/gun ownership. Omitted category is “other party wins.”
Standard errors in parentheses. ^p < 0.1, *p < 0.05, **p < 0.01, ***p < 0.001.
50
Table O3: Effect of Partisan Gridlock Relative to Ideological Gridlock
Intercept
Partisan Gridlock
Republican
Partisan Gridlock x Republican
Age
(1)
0.265***
(0.0236)
-0.119**
(0.0384)
-0.0985^
(0.0581)
0.178*
(0.0803)
—
(2)
0.405***
(0.061)
-0.167^
(0.095)
—
Study 1
(3)
0.174***
(0.0275)
-0.0263
(0.0419)
—
(4)
0.241***
(0.0238)
-0.0813*
(0.0364)
—
(5)
0.25***
(0.0316)
-0.0978^
(0.0515)
—
—
—
—
—
—
—
(6)
0.296***
(0.0287)
-0.0463
(0.0454)
-0.0137
(0.0583)
0.18^
(0.0979)
—
—
—
—
—
—
—
-0.00486**
—
(0.00167)
0.00304
—
(0.00253)
—
0.142***
(0.0409)
—
-0.0828
(0.0637)
—
—
Partisan Gridlock x Age
—
Under 30
—
Partisan Gridlock x Under 30
—
Non-White
—
Partisan Gridlock x Non-White
—
—
—
College Degree
—
—
—
-0.0158
(0.0528)
0.0891
(0.0856)
—
Partisan Gridlock x College Degree
—
—
—
—
136
0.0745
0.0535
144
0.0887
0.0691
144
0.113
0.094
144
0.0364
0.0157
N
R2
Adjusted R2
(7)
0.36***
(0.0666)
0.0177
(0.125)
—
(8)
0.257***
(0.0352)
0.0178
(0.0533)
—
Study 2
(9)
0.284***
(0.0284)
0.0178
(0.0473)
—
(10)
0.302***
(0.0355)
0.00625
(0.0654)
—
—
—
—
—
—
—
—
—
—
—
—
-0.00197
(0.00182)
-0.000752
(0.00346)
—
—
—
—
—
—
—
—
—
—
—
0.0729
(0.0498)
-0.0491
(0.081)
—
—
—
—
—
—
-0.0217
(0.0428)
0.0594
(0.0674)
143
0.0306
0.00971
—
—
—
0.042
(0.0594)
-0.0979
(0.09)
—
—
—
—
—
155
0.0296
0.0103
157
0.0134
-0.00598
157
0.0151
-0.00425
157
0.00802
-0.0114
Note: OLS regression of approval of how Congress is handling energy/gun ownership. Omitted category is “ideological gridlock.”
Standard errors in parentheses. ^p < 0.1, *p < 0.05, **p < 0.01, ***p < 0.001.
51
—
-0.0148
(0.0505)
-0.0211
(0.0839)
156
0.00291
-0.0168
Online Appendix 3: Issue Status Quo and Within Party Effects
Although the issues we examine clearly differ on the degree of cross-party
consensus (Egan, 2014), there may be other issue-level characteristics that could also
affect public responses to partisan conflict, and to gridlock in particular. For instance, the
current policy (status quo) on an issue could be relatively more favorable for one party
than the other, which could in turn affect preferences for action. To rule out the
possibility that our main result on preferences for action over gridlock is driven by
partisans’ consideration of status quo policy, we show that the negative effect of gridlock
relative to the opposing side winning holds within parties in study 1 (energy), and that the
null effect of gridlock relative to the opposing side winning holds within parties in study
2 (guns). If the position of the status quo were driving the results (i.e., if people have no
preference for “do something” politics and respond only to whether they are advantaged
by the status quo), we would expect to observe differently signed effects across parties on
an issue, as one side would favor gridlock to a victory by the opposing party. Instead, we
find that in both studies, the effects are similar for Democrats and Republicans, patterns
that are consistent with a preference for legislative action on consensus issues (but not
non-consensus issues). Moreover, these patterns holds if we restrict our analyses to
participants whose party matches their ideology (i.e., removing Democrats who selfidentify as conservative and vice versa), or to participants who correctly recalled the
party positions on the issue at hand.
Given the small number of Republicans in our study, we cannot rule out the
possibility that Republicans might actually favor gridlock over a victory by the opposing
party on gun policy (study 2). However, regardless of which party is favored on energy
52
policy (study 1), we would not expect to find that partisans from both parties favor a
victory for the opposing party over legislative gridlock unless people have a preference
for action that extends beyond partisanship. That is, the observed patterns on energy
policy cannot simply be explained by a current policy that favors one party over the
other.
Figure O1: Effect of Gridlock Relative to Other Party Wins, by Respondent Party
Note: Y-axis shows the effect of gridlock relative to a victory by the opposing party on
approval of how Congress is handling energy/gun ownership policy. ‘All Partisans’
includes all self-identified Democrats and Republicans (including partisan leaners).
‘Ideology Match’ includes only self-identified partisans who also provided an ideological
self-placement congruent with their party (e.g., Democrats who were very liberal to
moderate, Republicans who were moderate to very conservative). ‘Party Recall’ includes
only self-identified partisans who also correctly answered a question at the end of the
survey about which party took which position on the issue.
53
Online Appendix 4: Manipulation Checks and Attentiveness
We asked three manipulation check questions to gauge attentiveness. These
questions were all asked at the very end of the survey so as not to prime participants’
responses to the dependent variables. One set of questions gauged attention to the
treatment, while a third question gauged attention to the policy information that was
provided to all participants. The first treatment-based manipulation check asked
participants to recall what happened to the legislation that was debated in Congress (i.e.,
the outcome), with response options capturing compromise, a bill favoring Democratic
priorities passing, a bill favoring Republican priorities passing, or disagreement and no
bill passing. The correct answer varied by treatment assignment. For those participants
who said that no bill passed, the next manipulation check asked why Congress was unable
to reach consensus on the bill (i.e., the explanation). Response options were that neither
party was willing to sacrifice its ideological principles, neither party was willing to hand
the other side a victory in the run-up to the next election, and that no reason was given.
Again, the correct answer was contingent on treatment assignment. The policy
description question asked participants to recall which party favors government
incentives to promote the development of renewable energy technology (or favored
additional gun regulation) (i.e., party recall). Response options were Democrats,
Republicans, both parties, or neither party. The correct answer was Democrats in both
studies. Participants who answered these manipulation checks correctly spent
significantly more time reading the policy information (including their assigned
treatment) than did participants who answered incorrectly (Study 1: mean of 45 seconds
54
versus 35 seconds, p<0.001; Study 2: mean of 28 seconds versus mean of 21 seconds,
p<0.001).13
In an effort to be transparent about the how attentiveness affects the results (see
Berinsky et al., 2014), we plot the two primary gridlock effects across increasing levels
of attentiveness. As shown in Figure O2, the key patterns regarding “do something”
politics do not vary across levels of attentiveness. In study 1, gridlock has a negative
effect on evaluations (relative to the other party winning), pointing to a robust preference
for action. In study 2, there is a null effect across increasing levels of attention. By
contrast, the relationship between ideological and partisan gridlock is more sensitive to
respondent attentiveness (Figure O3). Here, partisan gridlock is viewed as significantly
worse than ideological gridlock only among respondents who could recall both the
outcome of the legislation and the explanation for gridlock, and among those who could
recall both of these items plus the party positions on the issue. In other words,
participants who failed to correctly recall the explanation provided for gridlock were
unaffected by said explanation.
We also investigated the possibility that restricting the analyses to those
participants who passed the manipulation checks skews the sample and introduces bias by
systematically excluding certain types of participants. To do so, we estimated logistic
regression models predicting passage of the checks in each study (see Table O4). As a
dependent variable, we use the same threshold of attentiveness used in the main analyses
reported in the text: correctly recalling both the outcome of the bill and the explanation
13
This excludes outliers who spent more than the 95th percentile of time on this page.
55
provided (see footnote 10). We use standard demographic and political variables as
predictors. In study 1 (energy), only age is associated with the outcome of the
manipulation checks: participants under 30 years old were less likely than older
participants to pass the checks than (p<0.1). Likewise, in study 2 (guns), younger
participants were less likely to pass the checks (p<0.05). Whites were more likely than
non-whites to pass the checks in study 2 (p<0.1). Although it is important to
acknowledge how the sample changes when we restrict analyses to participants who
passed the manipulation checks, we did not find evidence of heterogeneous treatment
effects on either age or race (Online Appendix 2), suggesting that this restriction is
unlikely to affect the generalizability of the findings.
Figure O2: Effect of Gridlock Relative to Other Party Wins by Passage of Manipulation
Checks
Note: Y-axis shows the effect of gridlock relative to a victory by the opposing party on
approval of how Congress is handling energy/gun ownership policy. ‘All Partisans’
includes all self-identified Democrats and Republicans (including partisan leaners).
‘Outcome Recall’ includes only partisans who correctly identified the outcome of the
legislation. ‘Outcome + Party Recall’ includes only partisans who correctly identified the
outcome of the legislation and who recalled the position of the Democratic Party.
56
Figure O3: Effect of Partisan Gridlock Relative to Ideological Gridlock by Passage of
Manipulation Checks
Note: Y-axis shows the effect of partisan gridlock relative to ideologically framed
gridlock on approval of how Congress is handling energy/gun ownership policy. ‘All
Participants’ includes both partisans and pure independents. ‘Outcome Recall’ includes
only respondents who correctly identified the outcome of the legislation. ‘Outcome +
Explanation Recall’ includes only respondents who correctly identified the outcome of
the legislation and who correctly identified the explanation for gridlock. ‘Outcome +
Explanation + Party Recall’ includes only partisans who correctly identified the outcome
of the legislation, the explanation for gridlock, and who recalled the position of the
Democratic Party.
57
Table O4: Logit Models Predicting Passing Manipulation Check (Studies 1 and 2)
(Study 1)
(Study 2)
*
Intercept
0.762
0.741^
(0.382)
(0.394)
Male
0.104
0.0269
(0.182)
(0.181)
White
0.0634
0.35^
(0.219)
(0.205)
Education
0.537
0.0493
(0.427)
(0.436)
Independent
-0.4
0.106
(0.252)
(0.208)
Republican
-0.327
0.268
(0.209)
(0.238)
Under 30
-0.314^
-0.383*
(0.18)
(0.181)
N
647
652
Log Likelihood
-382
-381
Note: Logit models of passing manipulation checks (1 if recalled outcome and
explanation correctly). Omitted category for partisanship is Democrat (modal response).
Standard errors in parentheses. ^p < 0.1, *p < 0.05, **p < 0.01, ***p < 0.001.
58
Online Appendix 5: Causal Mediation Analysis
This appendix provides the full mediation results discussed in the paper
(following Imai et al., 2011), as well as accompanying sensitivity analyses, which
estimate how much variance an omitted variable would need to explain in order to change
the sign of the mediated effect.
Table O5: Mediation Analysis for Study 1 (Energy)
Control
Treatment
ACME
ADE
Total Effect
Compromise Own Win
0.048
-0.026
0.021
(p<0.001)
(p=0.23)
(p=0.39)
Compromise Other Win
-0.079
-0.007
-0.086
(p<0.001)
(p=0.76)
(p<0.001)
Compromise Gridlock
-0.109
-0.045
-0.15
(p<0.001)
(p=0.08)
(p<0.001)
Other Win
Gridlock
-0.055
-0.011
-0.066
(p<0.001)
(p=0.62)
(p=0.01)
Note: The mediation analyses leverage the approach outlined by Imai et al. (2011). In
each case, the outcome of interest is confidence in Congress and the mediator is approval
of how Congress is handling energy. The average causal mediation effect (ACME) refers
to the effect of the treatment through the mediator, and the average direct effect (ADE)
refers to the effect of the treatment on the outcome through other mechanisms.
59
Figure O4: Sensitivity Analysis for Study 1 (Energy)
Note: The figures report sensitivity analyses from the mediation analyses showing how
much variance an omitted variable needs to explain in both the outcome and mediator
variables to overturn the results. The contours of each plot represent the values of the
average causal mediation effect (ACME) for different combinations of the mediation Rsquared and outcome R-squared values. The bold-faced line indicates the combinations of
variance explained by an omitted variable (in the outcome (Y) and mediator variable
(M)) that would lead to a change in the sign of the mediated effect.
60
Table O6: Mediation Analysis for Study 2 (Gun Ownership)
Control
Treatment
ACME
ADE
Total Effect
Compromise Own Win
0.016
-0.025
-0.009
(p=0.27)
(p=0.38)
(p=0.76)
Compromise Other Win
-0.078
-0.017
-0.096
(p<0.001)
(p=0.53)
(p<0.001)
Compromise Gridlock
-0.064
-0.065
-0.13
(p<0.001)
(p=0.01)
(p<0.001)
Other Win
Gridlock
0.015
-0.052
-0.036
(p=0.33)
(p=0.07)
(p=0.25)
Note: The mediation analyses leverage the approach outlined by Imai et al. (2011). In
each case, the outcome of interest is confidence in Congress and the mediator is approval
of how Congress is handling energy. The average causal mediation effect (ACME) refers
to the effect of the treatment through the mediator, and the average direct effect (ADE)
refers to the effect of the treatment on the outcome through other mechanisms.
61
Figure O5: Sensitivity Analysis for Study 2 (Gun Ownership)
Note: The figures report sensitivity analyses from the mediation analyses showing how
much variance an omitted variable needs to explain in both the outcome and mediator
variables to overturn the results. The contours of each plot represent the values of the
average causal mediation effect (ACME) for different combinations of the mediation Rsquared and outcome R-squared values. The bold-faced line indicates the combinations of
variance explained by an omitted variable (in the outcome (Y) and mediator variable
(M)) that would lead to a change in the sign of the mediated effect.
62
Supplemental Appendix References
Ahler, D. J. (2014). Self-fulfilling misperceptions of public polarization. Journal of
Politics, 76(3), 607-620.
Arceneaux, K. (2012). Cognitive biases and the strength of political arguments. American
Journal of Political Science, 56(2), 271-285.
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for
experimental research: Amazon.com's Mechanical Turk. Political Analysis, 20(3),
351-368.
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from
the workers? Making sure respondents pay attention on self-administered surveys.
American Journal of Political Science, 58(3), 739-753.
Bishin, B. G., Hayes, T. J., Incantalupo, M. B., & Smith, C. A. (Forthcoming). Opinion
backlash and public attitudes: Are political advances in gay rights
counterproductive? American Journal of Political Science. doi:
10.1111/ajps.12181
Carnes, N., & Sadin, M. L. (2015). The "mill worker's son" heuristic: How voters
perceive politicians from working-class families--and how they really behave in
office. Journal of Politics, 77(1), 285-298.
Christenson, D., & Glick, D. M. (2015). Chief Justice Robert's health care decision
disrobed: The microfoundations of the Supreme Court's legitimacy. American
Journal of Political Science, 59(2), 403-418.
63
Clifford, S., & Jerit, J. (2014). Is there a cost to convenience? An experimental
comparison of data quality in laboratory and online studies. Journal of
Experimental Political Science, 1(2), 120-131.
Dowling, C. M., & Wichowsky, A. (2015). Attacks without consequence? Candidates,
parties, groups, and the changing face of negative advertising. American Journal
of Political Science, 59(1), 19-36.
Grose, C. R., Malhotra, N., & Van Houweling, R. P. (2015). Explaining explanations:
How legislators explain their policy positions and how citizens react. American
Journal of Political Science, 59(3), 724-743.
Healy, A., & Lenz, G. S. (2013). Substituting the end for the whole: Why voters respond
primarily to the election-year economy. American Journal of Political Science,
58(1), 31-47.
Hersh, E. D., & Schaffner, B. F. (2013). Targeted campaign appeals and the value of
ambiguity. Journal of Politics, 75(2), 520-534.
Huff, C., & Tingley, D. (Forthcoming). “Who are these people?” Evaluating the
demographic characteristics and political preferences of MTurk survey
respondents. Research and Politics. doi: 10.7910/DVN/5TZWT2.
Imai, K., Keele, L., Tingley, D., & Yamamoto, T. (2011). Unpacking the black box of
causality: Learning about causal mechanisms from experimental and
observational studies. American Political Science Review, 104(4), 765-789.
Krupnikov, Y., & Levine, A. S. (2014). Cross-sample comparisons and external validity.
Journal of Experimental Political Science, 1(01), 59-80.
64
Leeper, T. J., & Mullinix, K. J. (2014). To whom, and with what effect? Parallel
experiments on framing. Paper presented at the Annual Meeting of the American
Political Science Association, Washington, D.C.
Mullinix, K. J., Druckman, J. N., & Freese, J. (2014). The generalizability of survey
experiments. Working Paper. Institute for Policy Research, Northwestern
University. Retrieved from
http://www.ipr.northwestern.edu/publications/papers/2014/ipr-wp-14-19.html
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon
Mechanical Turk. Judgment and Decision Making, 5(5), 411-419.
Ryan, T. J. (2014). Reconsidering moral issues in politics. Journal of Politics, 76(2), 380397.
Testa, P. F., Hibbing, M. V., & Ritchie, M. (2014). Orientations toward conflict and the
conditional effects of political disagreement. Journal of Politics, 76(3), 770-785.
Tomz, M. R., & Weeks, J. L. P. (2013). Public opinion and the democratic peace.
American Political Science Review, 107(4), 849-865.
65