‘J UST T HINKING :’ ATTITUDE D EVELOPMENT,
P UBLIC O PINION ,
AND
P OLITICAL R EPRESENTATION∗
Mathieu Turgeon
University of North Texas
[email protected]
September 6, 2007
This project was funded by the NSF through its Dissertation Improvement Grant program and through an
award from TESS (Time-Sharing Experiments for the Social Sciences). I would like to thank Adam Berinsky,
Matthew Eshbaugh-Soha, Jon Krosnick, James Kuklinski, Robert Luskin, Raul Madrid, Daron Shaw, John Sides,
and Don Vann for their insightful comments and suggestions.
∗
1
ABSTRACT
The role of public opinion polls in electoral democracy is undeniable because, for good or for
bad, they affect the kinds of laws and policies elected officials enact. But the voices measured in
polls are not perfectly representative of their populations of interest. More precisely, polls generally sing with a more “knowledgeable” accent than those they represent because of the greater
tendency of the less knowledgeable to remain silent. This distortion, however, can be palliated by providing conditions more propitious to attitude development. By relying on surveyexperiments conducted in Brazil and in the U.S., I present evidence that inducing people to
think more carefully before answering attitude questions reduces substantially the likelihood of
the less knowledgeable, which compose most of the Brazilian and American populations, to express a nonopinion response. Thus providing people with greater opportunity to think about
politics—something most of them do not do very frequently—makes for more representative
measures of public opinion. But the analyses also suggest that increased thought induces greater
uncertainty or ambivalence among the most knowledgeable. As a whole, this paper improves
our understanding on how people come to develop political attitudes and on the conditions that
lead to greater attitude uncertainty or ambivalence.
2
Polls are the worst way of measuring public opinion and public behavior, or of predicting elections—except for all of the others.
—Humphrey Taylor, chairman of The Harris Poll
Public opinion polls are generally more egalitarian than elections and referendums because
their samples are more representative than the fraction of the electorate who actually vote (Converse
1987a; Converse 1987b; Gallup and Rae 1940; Gosnell 1940; Verba 1996).1 But if generally more
representative than elections, polls are still imperfectly representative. One source of distortion
is sampling error: the discrepancy between sample and population (Ascher 2001). This may
stem, in part, from the sampling frame by excluding some kinds of people more than others—an
example of systematic sampling error.2 Random-digit-dialing techniques, for example, naturally
exclude anyone who does not possess a phone line thus underrepresenting the poor and the
young (Groves 1989; Schuman and Presser 1981).
A second source of distortion is unit nonresponse. The designated respondents may never be
reached, may decline to be interviewed or may fail to complete the interview. The actual sample
refers to all the selected respondents while the effective sample refers only to those who agreed
to complete the interview. The latter is less representative than the former, but, again, there exist
ways to reduce these discrepancies like increasing callbacks and providing respondents with
incentives to complete the interview.
A third source of distortion is nonopinion response or item nonresponse.3 The poor, the
young, the less educated, women, and African Americans give more nonopinion responses than
more affluent and older educated people, males, and whites (Bishop, Tuchfarber and Oldendick
1986; Francis and Busch 1975; Schuman and Presser 1981). The sharpest gradient, however, is
with knowledge: the less knowledgeable give more nonopinion responses than the more knowledgeable (Althaus 1996; Althaus 2003; Ehrlich 1964; Faulkenburry and Mason 1978; Krosnick
and Milburn 1990; Wright and Niemi 1983). As a result, some people receive less weight in polls.
1
I use poll and survey interchangeably to refer to any study aiming to measure people’s attitudes and/or behaviors by questioning samples of relevant publics.
2
Systematic sampling error is distinct from random sampling error in that we know the source of the discrepancies. Random sampling error could only be corrected by drawing repeated samples of the same population, a very
expensive enterprise.
3
Nonopinion responses or item nonresponses refer to the expression of ‘don’t knows’ or refusals to answer a
question.
3
The effective sample is now reduced to the “measured” sample which refers only to respondents
who provide opinions—an even lesser representative sample.
Improving the representativeness of opinion surveys is imperative for anyone interested in
ameliorating electoral democracy. Indeed, public opinion polls serve democracy by sending
messages to elected officials about what people want and do not want, and elected officials generally respond with tailored policies (Geer 1996; Quirk and Hinchliffe 1998; Stimson, MacKuen
and Erickson 1995).4 Even the U.S. Supreme Court is thought to follow public opinion in its decisions (McGuire and Stimson 2004; Mishler and Sheehan 1993). But the democratic value of these
ensuing public policies and court decisions all depend on how well, in the first place, opinion
polls represent the people’s voice.
In this paper, I argue that scholars have overlooked, or failed to properly study, the role of
simply thinking on attitude development. More precisely, I argue that thought given to knowledge already held or recently activated promotes attitude development because it affects the
cognitive processes people go through when developing an attitude, most notably the number
of considerations one brings to mind. With more considerations to rely on, people can more
easily develop an attitude which reduces, in the end, their likelihood to express a nonopinion.5
Thus while this paper addresses the important question of polls’ representativeness and the
ensuing consequences on the democratic process, it also provides insight into the broader question of how people develop political attitudes. Understanding how people develop attitudes is
of great relevance to political scientists because of the role attitudes play in explaining political
behavior or more general political predispositions.
The results of two original survey-experiments show that thought, similarly to knowledge,
helps people develop poltitical attitudes, but also induces some to become more uncertain or
ambivalent about their political attitudes. Experimentation here has its usual advantages for
isolating causal effects (Kinder and Palfrey 1993; Green and Gerber 2002; McDermott 2002), and
more importantly for present purposes, allows for the manipulation of thought. The first surveyexperiment was conducted in the fall of 2004 with an urban sample of 1054 Brazilians, and the
second in the summer of 2005 with a national sample of 760 Americans.
4
But see Jacobs and Shapiro (2000) for a different perspective.
It is worth noting that in another paper I explore the effects of thought on the attitudes expressed per se. More
precisely, I examine whether increased thought helps people hold attitudes that are more reflective of their underlying interests. But the focus here is on attitude development which is why I examine nonopinion responses.
5
4
Finally, it is worth noting that previous work in political and social psychology has been
criticized for being conducted exclusively in the United States or other industrialized countries (Amir and Sharon 1987; Conover and Searing 2002; Moghaddam, Taylor and Wright 1992).
Conducting studies in both the U.S. and Brazil is a strength because it allows me to consider
whether the results vary cross-culturally or may be universal (Moghaddam, Taylor and Wright
1992; Nisbett 2003). Thus this paper also makes an important contribution to the nascent field of
comparative political psychology (Conover and Searing 2002).
Thinking and Attitude Development
Depending on the attitude object, some, many, or most people do not possess an attitude about
it. Thus many respondents decline to answer survey questions. Instead, they say ‘don’t know’
or simply refuse to say anything. Some of those nonopinion responses are true reflections of
the absence of an attitude toward a particular object. Some others are simply the refusal to
publicly express an attitude despite having one. This is particularly true for sensitive issues like
race where some respondents may prefer not to voice a preference that is not socially acceptable
(Berinsky 1999; Berinsky 2004; Tourangeau, Rips and Rasinski 2000).
In practice, however, it is impossible to make the distinction between those that truly do
not have an attitude from those that have one but prefer not to voice it. But it is reasonable to
believe that attitude expression is a function of attitude development. As attitude development
increases, nonopinion responses should decrease.
Attitude development (or formation, as often referred in the literature) is simply a special case
of attitude change. Indeed, while attitude change is understood as the passage from holding one
particular attitude to another one, like initially favoring euthanasia to later opposing it, attitude
development is the passage from not holding an attitude about a particular object to holding
one.
The issue is what affects attitude development. Some of the answers may be found in the
literature on question order and wording effects (for good reviews see Schuman and Presser
1981; Schwartz and Sudman 1992). One possibility is rapport: the respondent’s feeling “more
relaxed, trusting, or committed as an interview proceeds (Schuman and Presser 1981: 50).” Thus
5
Sigelman (1981) finds respondents more willing to answer the presidential popularity question
if asked late in the questionnaire.
Another possibility is whether the question offers a ‘don’t know’ option. There is ample
evidence that ‘don’t know’ options in survey questions (commonly referred to as “nonopinion
filters”) substantially increase nonopinion responses (Ehrlich 1964; Schuman and Presser 1981).
Nonopinion filters produce more nonopinions because they promote ‘satisficing’ over ‘optimizing’ (Krosnick 1991; Krosnick 2002). According to Krosnick, many respondents possess attitudes
that are not totally developed but for which respondents have enough ingredients to develop
one, given certain efforts like retrieving considerations related to the issue at hand and integrating them into a response, a process Krosnick refers to as ‘optimizing.’ But, depending on
the respondents’ cognitive ability, interest, familiarity, and engagement with the issue at hand,
respondents sometimes find it easier to simply shortcut the whole cognitive process and select
‘don’t know’ as an answer, i.e. ‘satisfice.’ The focus here is about this sort of cognitive effort.
Krosnisk’s dual-path model to the survey response is similar to other well-known dualprocess models of attitude development or change like the Elaboration-Likelihood Model (ELM)
(Petty and Cacioppo 1986a; Petty and Cacioppo 1986b) and the Heuristic-Systematic Model
(HSM) proposed by Chaiken and colleagues (Bohner, Moskowitz and Chaiken 1995; Chen and
Chaiken 1999; Chaiken, Liberman and Eagly 1989; Eagly and Chaiken 1993). At the core, these
models posit that people adopt strategies from the low and high ends of a continuum of processing effort depending on their motivation and cognitive ability. Highly motivated and cognitively
able people generally favor a more effortful process (central route or systematic processing) than
less motivated and cognitively able ones that tend rather to employ the peripheral route to information processing or rely on heuristics (information shortcuts), although this may not always be
the case, especially with respect to the HSM where motivated and cognitively able people may
also rely, at times, on heuristics to make judgments. Note that this is also true, to some extent, of
the ELM (Petty and Wegener 1998).
When employing the effortful path to information processing, people put great cognitive efforts at retrieving attitude-relevant considerations, scrutinizing their relevance and importance,
and integrating them into newly developed or revisited attitudes. Effortless information processing, on the other hand, has people rely on simple heuristics easily retrieved from memory or
6
provided by the environment to develop or revisit an attitude.
But more importantly for present purposes, these dual-process models, including Krosnick’s,
constitute the basis for understanding the effects of thought on attitude development. Indeed,
thought promotes attitude development because it affects the balance of cognitive efforts deployed into developing an attitude by pushing it toward the high end of the processing effort
continuum. More specifically, thought increases the opportunities for effortful processing because it provides people with more time (and also motivation) to perform the deliberating tasks
of more systematic information processing, including, most notably, comprehension of the attitude object, retrieval of attitude-related considerations from memory, and integration of these
retrieved considerations into an attitude. And, this is particularly true in the survey environment where room for deliberation is rather thin knowing that the average response time is less
than five seconds (Bassili and Fletcher 1991; Tourangeau, Rasinski and D’Andrade 1991)!
First, thought increases comprehension of the attitude object (Flanders and Thistlethwaite
1967). In the survey environment, difficult or badly worded questions may elicit more nonopinion responses by making it hard to tell what information is being sought. More motivation and
enough time to think improve respondents’ comprehension by increasing attention to the question and any accompanying instructions, decreasing, ultimately, the expression of nonopinion
responses.
Second, thought assists the retrieval of considerations by inducing effortful search of relevant
bits of information from long-term memory (Sadler and Tesser 1973). With more considerations
in mind, the easier it should be to develop an attitude because considerations constitute, after
all, the main ingredients of attitudes (Zaller and Feldman 1992).
Note that this is consistent with Zaller’s belief-sampling model which stipulates that most
respondents, when asked to express an attitude in a survey, make up a quick answer, highly
influenced by the most salient considerations they have in mind (Zaller 1992; Zaller and Feldman
1992). Thought affects the sampling of considerations by expanding the number of considerations
retrieved.
Third, and finally, thought facilitates the integration of the retrieved considerations into an
attitude by providing people with more time and motivation to sort them out (Fazio 1990; Petty
and Cacioppo 1986a; Eagly and Chaiken 1993; Sanbonmatsu and Fazio 1990).
7
But the effects of thought on attitude development are conditioned by knowledge of the attitude object. The most knowledgeable, by definition, have the most considerations to recall
(Sadler and Tesser 1973), which should increase thinking’s effect, but also tend to have welldeveloped attitudes already (Althaus 1996; Althaus 2003; Fishhoff 1991; Alwin and Krosnick
1991), which should reduce it. The less knowledgeable, on the other hand, are less likely to have
well-developed attitudes already and should, therefore, benefit the most, as long as they can
retrieve at least some considerations. In that sense, thought should not affect much those that
are totally ignorant about the attitude object because they have essentially nothing to think about
except for maybe a few considerations there and there that do not require great deliberation.
The provision of domain-specific information, which also stirs some thought, also promotes
attitude development. Providing people with information increases the number of considerations that can be used to develop an attitude both directly or indirectly, by triggering the retrieval of relevant considerations already held. This too should benefit the less knowledgeable
most. Then again the provision of information may hinder the expression of an attitude for those
already holding one because new and convincing information contradicting an original attitude
may induce uncertainty or ambivalence (Alvarez and Brehm 1995; Alvarez and Franklin 1994).
To be sure, information helps develop attitudes, but it may also reduce attitude expression by
making some respondents more uncertain or ambivalent, especially among the most knowledgeable who have more considerations to recall (Barker and Hansen 2005; Tetlock 1993). That,
of course, may also be true of thought as shown by Linville (1982).
Study 1: Support for Government Spending in Brazil
The Brazil study was conducted by Market Analysis Brazil.6 Respondents were selected from
ten large Brazilian cities scattered around the country, and the interviews were conducted by
telephone. Thus the sample is not a perfect national sample but close enough as Brazil’s urban
population now accounts for over 80% of Brazil’s population, and these ten cities harbor, themselves, over 28% of the total population. The sampling follows a proportionate to population
size distribution, with stratified clustering of census districts and sectors within each city, and
6
www.marketanalysis.com.br
8
random selection of districts, clusters, and household. The sample frame includes all adults 1869 years old from the general population, and quotas of gender, age, and socio-economic level
are followed to ensure representativeness.7 The response rate was 29.1%.8
The experimental manipulation involves five treatment groups and a control group, all randomly assigned. Each group was asked one question about their preferred level of government
spending and five related knowledge questions, but with differences in wording, ordering, and
surrounding script to manipulate information and thought.
The control group was simply asked these questions in an unadorned, conventional way,
with the spending question preceding the knowledge questions. The first treatment group (ST,
for “stop-and-think”) was asked the same questions in the same order but was also asked to
think for 30 seconds before answering the spending question. A still longer period for thinking
might produce stronger effects (Tesser and Conlee 1975), but 30 seconds was the longest pause I
felt I could afford. Note that the interviewers were barred from recording any preference before
the 30 seconds elapsed.
The second treatment group (FLAG, for flagging considerations) received the same questions
as the control group but with the order of the spending and knowledge questions reversed. The
exposure to the knowledge questions flags the factual information they ask about as relevant
considerations and stirs some thought. The third group (ANSWERS, for providing the correct
answers to the knowledge questions) received the same questions in the same order as in the
FLAG treatment (i.e., knowledge questions before spending question), but respondents were
also told the right answers to each knowledge question after answering it. The fourth treatment
(ST-FLAG) was the same as FLAG, except that the subjects were also asked, just before answering
the spending question, to stop and think about the issue before choosing their preferred level of
spending, just as in the ST treatment. The fifth treatment (ST-ANSWERS), similarly, is the same
as ANSWERS, except that the subjects were also asked, just before answering the policy question,
to stop and think, again just as in the ST treatment.
The spending item asks first whether government spending should be increased, decreased,
7
Socioeconomic level follows the standard Brazilian criterion classification (Associação Nacional de Empressas de
Pesquisa (ANEP), www.anep.org.br.
8
As measured by AAPOR’s RR5: the "number of complete interviews divided by the number of interviews
(complete plus partial) plus the number of non-interviews (refusal and break-off plus non-contacts plus others)
(American Association for Public Opinion Research 2004).
9
or kept as is. Those respondents wanting to increase or decrease it are then asked by how much:
a little, somewhat, or a lot. Note that the government spending question does not offer a ‘don’t
know’ option to promote ‘optimizing’ over ‘satisficing,’ as suggested by Krosnick (2002). There
are also five multiple-choice knowledge questions, all domain-specific. Since the knowledge
questions are asked to provide as well as measure knowledge, it is worth noting that the facts
being asked about are balanced, in the sense that some might be expected to make most people
want to spend less, others to make most people want to spend more. The knowledge questions
asked about items on which governments in Brazil spend the most (social security) and the least
(culture), about where most of the government’s revenues come from (corporate income tax),
about the evolution of the national debt (it has decreased), and about the consequences of the
recent social security reform (the reform is likely to reduce the costs of the program). The order
of the knowledge questions was randomized. Market Analysis Brazil also collects information on
socio-demographic variables including age and education.
Thus a total of six questions were asked of each of six groups (one control and five treatments), each numbering about 175 respondents. The appendix provides the question wording
for all groups.
Note that the stop-and-think instruction proposed here differs from the manipulation in Zaller and Feldman (1992). Their design, borrowed from Wilson and colleagues (Wilson, Sunn,
Bybee, Hyman and Rotondo 1984; Wilson, Dunn and Lisle 1989; Wilson, Kraft and Dunn 1989),
has the respondents required not only to stop and think but to voice their thoughts as they
do so. Zaller and Feldman call their manipulation “stop-and-think,” but it is really “stop-andthink-aloud.” The voicing of relevant thoughts makes this manipulation an impure measure of
the effect of thought alone. Interviews are social interactions, and the thoughts respondents
choose to express are not necessarily an unbiased sample of the thoughts they have. Thus I
stimulate thought only by requesting respondents to spend 30 seconds thinking like done by
Tesser and his colleagues in their studies exploring the effects of thought on self-induced attitude change (e.g. Tesser and Conlee 1975; Tesser and Cowan 1975; Tesser 1976; Tesser and
Leone 1977; Tesser 1978; Millar and Tesser 1986).9
9
It is also worth noting here that Zaller and Feldman’s (1992) experimental design does not allow to test for the
hypothesis that thought promotes attitude development because their “stop-and-think” treatment subjects were not
offered nonopinion filters but their control group subjects were. As mentionned earlier, nonopinion filters encourage
nonopinion responses, preventing altogether the comparison of both groups.
10
The design proposed here also diverges from the “Deliberative Polls” conducted by Fishkin
and Luskin (e.g., Fishkin, 1991 and 1997; Fishkin and Luskin, 1999; Luskin and Fishkin, 1998;
Luskin, Fishkin, and Jowell 2002). In their experiments, randomly selected participants gather
over a weekend period and are provided with objective and balanced information on a targeted
issue. During the weekend, participants are also required to break into group discussions. Their
deliberation design, therefore, differs from mine in that participants also interact with other participants. What I propose here is deliberation "from within" (McGuire 1960, 2000: 191). Moreover,
their studies are more concerned with attitude change than attitude development.10
More in line with what I want to show is the work by Fournier and Turgeon (unpublished)
that finds respondents to be more likely to express a vote intention if asked late in the questionnaire. The authors label this a “deliberation” effect. In a split-sample experiment where half the
respondents received a vote intention question early in the questionnaire, and the other half late,
late-answering respondents were more likely to express a vote intention than early-answering
ones, presumably because they had been stimulated by the earlier questions to think about some
relevant considerations. In that respect, the result by Sigelman (1981) that finds respondents
more likely to answer the presidential popularity question if asked late in the questionnaire may
also be explained by a “deliberation” effect.
The treatments are expected to affect cognitive processing and thus the likelihood of expressing a nonopinion. Recall that the experiment stimulates thought in three different ways.
First, the stop-and-think instruction in ST, ST-FLAG, and ST-ANSWERS is intended to increase
thought by motivating respondents to think and giving them more time to do so before answering the spending question. Second, asking the knowledge questions before the spending
question in FLAG, ANSWERS, ST-FLAG, and ST-ANSWERS serves to highlight some relevant
empirical considerations and thus to stimulate thought. Third, ANSWERS, and ST-ANSWERS
provide information, activate considerations, and stimulate thought. Finally, note that ST-FLAG
and ST-ANSWERS benefit from two and three of the stimuli to think, respectively.
As a consequence, attitude development fostered by thought should manifest itself empirically by
significantly reducing the expression of nonopinion responses. But, for reasons mentioned above, the
effect should be particularly strong for the less knowledgeable people. Thus, as compared to the
10
In addition, most of their events are really only quasi-experiments with “control groups” not quite separated
by random assignment.
11
control group, the less knowledgeable subjects in the treatment groups should be less likely to express
nonopinion responses. Because subjects in ST-FLAG and ST-ANSWERS benefit from two and three
of the stimuli to thought, respectively, the effects should be strongest in these two groups.
But, as argued earlier, increased thought and information may also lead to greater uncertainty or ambivalence, especially among the most knowledgeable subjects who have many more
considerations to retrieve. Increased uncertainty or ambivalence would manifest itself empirically by increasing the probability of expressing a nonopinion response. Therefore, as compared
to the control group, the most knowledgeable subjects in the treatment groups should be more likely to
express nonopinion responses. But the effects may also vary by treatment group. More precisely,
the effects should be weakest for subjects in ST because they are not provided with any new
information or considerations worth thinking about. Increased thought, for those subjects, may
only result into retrieving more redundant or consistent considerations with their initial attitude
(Tesser 1976; Tesser 1978). Subjects in the other treatment groups, however, may find it harder to
ignore some of the considerations activated that are orthogonal or inconsistent with their initial
attitude. This is particularly true of subjects who actually receive information. As a consequence,
among the most knowledgeable subjects, those in ANSWERS and ST-ANSWERS should be the
most likely to express nonopinion responses, followed by those in FLAG and ST-FLAG.
Explaining Nonopinion Responses on Government Spending
Table 1 presents the percentages of subjects expressing nonopinion responses, as defined here as
the sum of ‘don’t knows’ and middle-category responses. Middle category responses (No change
in either spending or taxes) are considered here as nonopinions because this category is known to
be a safe harbor for “nonattitudinal” respondents (Converse and Pierce 1986), especially when
nonopinion filters are not offerred (Ehrlich 1964), just like here. Moreover, Krosnick (1991), for
example, writes that “interviewers sometimes explicitly discourage ‘don’t know’ responses by
pressing respondents to choose a substantive answer to the question (p.220).”According to informants in the polling industry, their reluctance to accept nonopinions as responses is justified
by their need to satisfy their clients, who expect completed interviews to include few nonopinionated respondents.11 Thus if the interviewers are reluctant to accept nonopinion responses as
11
Personal communications with market researchers in both the U.S. and Brazil.
12
valid answers and nonopinion filters are not offered, then it is quite plausible that true nonopinion holders would have chosen the middle category instead, especially when the question label
identifies clearly the middle category as the status quo like in the present case. Admittedly, there
ought to have some meaningful middle-category responses, but we should not expect them to
vary much by treatment group.
From Table 1, we find, as expected, that the largest percentage of nonopinion responses is,
by far, found in the control group. Nonopinion responses average only 33.5% in the treatment
groups (pooled treatment) as compared to 41.8% in the control group. Thus thought reduced, on
average, the perentage of nonopinions by 8.3 percentage points across all treatments, a drop of
nearly 20% in nonopinions. Three of the five treatments (FLAG, ANSWERS, and ST-FLAG) and
the pooled treatment group have significantly fewer nonopinion responses (p<.05, one-tailed).
Note that the two other differences are both on the verge of being statistically significant. Contrary to expectations, ST-FLAG and ST-ANSWERS did not produce stronger effects. But the differences between the treatment groups is very small. Indeed, the average percentage difference
is only 0.7 between all groups. And the largest difference is found between FLAG (32.7%) and
ST-ANSWERS (34.7%) and reaches only 2.0%! The similarity of the findings between the treatments is striking and strongly suggest a common effect for thought and information in reducing
nonopinion responses, irrespective of how they are induced.12
[Table 1 about here.]
The results, so far, strongly suggest an important role for thought in affecting nonopinion
responses. But other factors like education and age have also been found to affect nonopinion responses (Bishop, Tuchfarber and Oldendick 1986; Francis and Busch 1975; Schuman and
Presser 1981). More educated and older people tend, on average, to express fewer nonopinions.
But more importantly, knowledge, as argued earlier, should also affect nonopinion responses and
condition thought’s effect. Thus I present a model explaining nonopinion responses on government spending that account for those factors.
12
It is worth noting that when nonopinions are defined only as the sum of ‘don’t knows,’ the percentage giving
nonopinion responses is lowest in ST-ANSWER (6.6%). The mean percentage in all the treatment groups together
is 8.1% versus 8.5% in the control group. In ST-FLAG, the percentage of nonopinions is also lower than in the
control group, in ANSWER it is the same, and in ST and FLAG it is slightly higher. But none of these differences is
statistically significant. But, again, because nonopinion filters were not offered here, it is hard to be fully confident
in these results.
13
Recall first that the experimental design has five treatment groups, each with different thought
stimuli or different combination of thought stimuli. Consider in this light a regression analysis
with three dummy variables, one for each stimulus to thought: 1) the stop-and-think prompt
labeled ST (in treatments ST, ST-FLAG, and ST-ANSWERS); 2) the activation of considerations
induced by asking the knowledge questions before the government spending question labeled
F LAG (in treatments FLAG, ANSWER, ST-FLAG, and ST-ANSWERS); and, 3) the provision of
domain-specific information labeled ANSW ERS (in treatments ANSWERS and ST-ANSWERS).
The interactions of ST with F LAG and ST with F LAG and ANSW ERS are also included to
account for subjects who were assigned to treatments ST-FLAG and ST-ANSWERS, respectively.
Finally, the variables Age, Education, and Knowledge (simply labeled as K in the equation)
complete the model. I estimate the following equation by maximum likelihood:
P (Yi = 1) = f (β0 + β1 ∗ Agei + β2 ∗ Educationi + β3 ∗ Ki + β4 ∗ Ki2
+β5 ∗ STi + β6 ∗ Ki ∗ STi + β7 ∗ Ki2 ∗ STi
+β8 ∗ F LAGi + β9 ∗ Ki ∗ F LAGi + β10 ∗ Ki2 ∗ F LAGi
+β11 ∗ ANSW ERSi + β12 ∗ Ki ∗ ANSW ERSi + β13 ∗ Ki2 ∗ ANSW ERSi
(1)
+β14 ∗ STi ∗ F LAGi + β15 ∗ Ki ∗ STi ∗ F LAGi + β16 ∗ Ki2 ∗ STi ∗ F LAGi
+β17 ∗ STi ∗ F LAGi ∗ ANSW ERSi + β18 ∗ Ki ∗ STi ∗ F LAGi ∗ ANSW ERSi
+β19 ∗ Ki2 ∗ STi ∗ F LAGi ∗ ANSW ERSi )
where f (X) = 1/(1 + e−X ). The βs are coefficients to be estimated, and the subscript i identifies
the individuals.
The dependent variable takes on the value of 1 for nonopinion responses defined as ‘don’t
knows’ or middle-category answers and 0 otherwise, for reasons discussed above.
Knowledge is measured by summing the number of correct answers to the five knowledge
questions. Correct answers were given a value of 1 and incorrect ones, ’don’t knows,’ and refusals a 0.13 The equation also includes Knowledge and Knowledge2 and their interactions with
ST , F LAG, ANSW ERS, ST ∗ F LAG, and ST ∗ F LAG∗ ANSW ERS to allow for thought’s effects
13
To the five knowledge questions, 28.7% of the subjects got them all wrong, 30.3% answered one correctly, 23.7%
answered two correctly, 12.0% answered three correctly, 4.7% answered four correctly, and .7% answered all the five
questions correctly.
14
to differ by knowledge in ways explained earlier. In that respect, the inclusion of Knowledge2
captures the curvilinear conditioning effects of knowledge on thought. Finally, note that this
model entails a lot of coefficients but not many variables. It is the interactions of these variables
in the product terms that make for the numerous coefficients.
The results from the logit model are reported by treatment group in Table 2. Contrary to
expectations, older and more educated people are not more likely to express fewer nonopinion
responses. Knowledge and thought, on the other hand, appear to affect nonopinion responses as
the significance of both Knowledge and Knowledge2 for subjects in the control group attest. But
knowledge also appears to condition thought’s effects, as suggested by the daggers that identify
treatment group coefficients that are statistically different from those in the control group. The
effects of knowledge and thought, however, can hardly be interpretable from Table 2 because
of the intrinsic difficulty of interpreting logit coefficient estimates. Thus to visualize better the
effects of knowledge and thought, Figure 1 presents, for each treatment mapped against control,
the probability of expressing a nonopinion response for given levels of knowledge, holding age
and education at their means.14
[Table 2 about here.]
[Figure 1 about here.]
The graphs in Figure 1 show that knowledge affects nonopinion responses (in most cases nonlinearly), but in different ways for the control and treatment groups. For the control group, an
increase in knowledge first increases the probability of a nonopinion response and then decreases
it as knowledge increases further up to a point where the probability to express a nonopinion is
almost zero. For subjects in FLAG, ANSWERS, and ST-ANSWERS, the probability to express a
nonopinion is first reduced with knowledge and then increased as knowledge increases further.
In ST, nonopinion responses decrease with knowledge. And, finally, in ST-FLAG, they increase
slightly with knowledge.
But despite the slight differences on the conditioning role of knowledge, the graphs suggest
that thought alone (ST), the activation of considerations alone (FLAG), information alone (AN14
The probabilities have been calculated using the statistical software Clarify which also computes standard errors to test for significant differences under various scenarios (Tomz, Wittenberg and King 2003; King, Tomz and
Wittenberg 2000).
15
SWERS), and the combination of either thought and activation of considerations (ST-FLAG) or
thought, activation of considerations, and information (ST-ANSWERS) all reduce, with varying
degrees, the probability of expressing a nonopinion for the less knowledgeable people.
In every treatment, the thought stimuli had essentially no significant effect on the probability to express a nonopinion for the least knowledgeable (score of 0 on the political knowledge
scale).15 This result is in line with what was expected: thought should not affect those that know
essentially nothing about a particular attitude object. The effects are strongest, as hypothesized,
among those that do not know much about government spending but a bare minimum (scores
of 1 and 2). Inducing those subjects to think more carefully and/or providing them with information reduced substantially their probability to express a nonopinion response. The differences
are significant at .05 (one-tailed) in every treatment for all subjects who scored 1 on knowledge,
and also for those who scored 2 in FLAG or ANSWERS. The three other differences are all very
close of being statistically significant too. Contrary to expectations, again, the strongest effects
are those found in FLAG, and not in ST-FLAG and ST-ANSWERS, where the probability of expressing a nonopinion was reduced by .24 and .21 in the former group as compared to the control
for subjects who scored 1 and 2 on the knowledge scale, respectively. The weakest effects, but
still substantial in size, are found in ST where the probability to express a nonopinion was reduced by .16 and .14 as compared to the control for subjects who scored 1 and 2 on knowledge,
respectively. Asking people simply to think more carefully reduces nonopinion responses but it
appears that they do even better when provided with some considerations worth thinking about.
The size of the effect of thought on the probability to express a nonopinion for the less knowledgeable is substantial. On average, inducing thought reduces the probability to express a
nonopinion response by .15 for those people who are known to be the most silent in opinion
polls (Althaus 1996; Althaus 2003; Ehrlich 1964; Faulkenburry and Mason 1978; Krosnick and
Milburn 1990; Wright and Niemi 1983).16 This result is important given the size of that group in
the population (they make up here for 82.7% of the total sample). Thought reduces nonopinion
responses for those people and makes, in turn, for more representative opinion polls.
15
The largest effect is found in ST-FLAG were the probability to express a nonopinion was .15 lower than that
in the control group but all the other differences hoover around .05. None of these difference, including that in
ST-FLAG, reaches statistical significance.
16
This probability was calculated by averaging the differences between all treatments and control for all the subjects who scored 2 or less on knowledge, including those who scored 0.
16
As predicted by the theory, the effects of thought ultimately disappear as knowledge increases further up to a point (scores of 4 and above), where more thought actually increases the
probability of expressing a nonopinion. In all treatment groups, except ST, subjects who scored
5 on the knowledge scale were significantly more likely, at .05 (one-tailed), to express a nonopinion. Those who scored 4 in ANSWERS and ST-FLAG also had a significantly higher probability
of expressing a nonopinion. The strongest effect was found for subjects in ANSWERS, as initially
expected, where the probability to express a nonopinion was .30 and .48 higher for subjects in
that group than for those in the control who scored 4 and 5 on knowledge, respectively. The
effect was also very strong in FLAG, ST-FLAG, and ST-ANSWERS. And as expected, the weakest
effect was found in ST where the probability to express a nonopinion was only .10 and .11 higher
than those in the control who scored 4 and 5, respectively.
As noted earlier, the most knowledgeable may be reaching a state of “analysis paralysis”
when explicitly asked to think extra carefully or consider new information making them, in
turn, become more uncertain or ambivalent. But, to be sure, this last result does not suggest that
thought reduces attitude development for the most knowledgeable. More knowledgeable people, as the results for the control group show and as documented elsewhere, generally possess
political attitudes on most issues. The extra thought prompts them instead to revisit their attitude about government spending, initiating a quest for the “correct” answer. The most knowledgeable become more ambivalent or uncertain because more thought allows for the retrieval of
numerous and possibly competing considerations. This is similar to results reported in Linville
(1982) and Tetlock (1993) that find cognitively able people to become more perplexed and moderate when induced to process orthogonal information. But the strongest similarity is found in
Barker and Hansen (2005) that find more knowledgeable voters to be less inclined to vote when
explicitly encouraged to process more systematically information, and thus think more carefully,
about political candidates.
These differences by knowledge groups discussed above are interesting and suggestive, but
it is needed to put them together to get to the overall effect of thought on the probability to
express a nonopinion. Indeed, the overall effect of thought on nonopinions can be calculated,
for each treatment group, by weighting its effect by the importance of each knowledge group in
17
the general population.17 By doing so, we find that inducing thought reduces, on average across
all treatments, the probability to express a nonopinion response by .10, holding, again, age and
education at their means. The size of this effect is important because it reduces the probability
to express a nonopinion by nearly 25% on average, from .44 to .34. Note also that the treatments,
here too, all show very similar effects, with the strongest found in FLAG (-.12) and the weakest
in ST-ANSWERS (-.09). The average difference between the treatments is only .01. In sum, these
results conform to the precedent ones and confirm, once again, the role of thought in reducing
substantially nonopinion responses.
Finally, it is worth noting here that the results do not suggest a different role for information
as compared to mere thought. Information, like thought, reduces nonopinion responses for the
less knowledgeable but also tends to produce attitude ambivalence or uncertainty among the
most knowledgeable. And overall, information reduces nonopinion responses too.
But how well do the results uncovered here travel to other issues and other contexts? Do they
vary cross-culturally and by issue or have more universal properties. In that light, another study
was conducted in the United States to answer this question.
Study 2: The Role of the Federal Government in Health Care in
the United States
A second experiment was designed and conducted in the U.S. by the on-line survey firm Knowledge Networks to look at the effects of thought on nonopinion responses.18 This study provides for
some improvements over the Brazil one. First, the U.S. study has a more representative sample,
as subjects were randomly selected from the entire U.S. adult population.19 Second, this study
17
The formula for the overall effect of thought for treatment j is
Tj =
5
X
(pji − pci )si
i=0
where pji and pci are the probabilities to express a nonopinion response for subjects who scored i on the knowledge
scale and are in treatment group j and the control, respectively. si represents the sample’s proportion of subjects
who scored i on knowledge. i runs from 0 (no correct answer to the knowledge questions) to 5 (all correct answers).
18
This study was conducted through TESS (Time-Sharing Experiments for the Social Sciences), by the National Science
Foundation.
19
The response rate was 30.9% as measured by AAPOR’s RR3: the “number of complete interviews divided by
the number of interviews (complete plus partial) plus the number of non-interviews (refusal and break-off plus noncontacts plus others) plus all cases of unknown eligibility. It also estimates what proportion of cases of unknown
18
offered subjects a nonopinion filter. The rationale for the inclusion of a ‘don’t know’ option was
to put the hypothesis that thought promotes attitude development to a harder test because, as
discussed earlier, such options generally encourage respondents to expend less cognitive efforts.
Third, and finally, the U.S. study was self-administered, avoiding altogether the pressure to provide interviewers with answers to questions for which subjects do not have an opinion. There is
one shortcoming. The U.S. study has fewer treatment groups.20 But this shortcoming is minor as
the results from Brazil show similar effects on nonopinion responses in each treatment group.
The experimental design involves two treatment groups and a control, all randomly assigned.
Each subject was asked a question about his or her preferred level of federal government intervention in the provision of health care services.
Subjects in the control and first treatment groups were also asked questions measuring their
knowledge regarding health care in the U.S. The questions measured the respondents’ knowledge about the percent of Americans without health insurance whatsoever (15%), about the percent of the federal government’s budget devoted to health care services (22%), about the fiscal
burden of a universal health care system (the fiscal burden would be highest for average and
above average income people), about its waiting period as compared to a private one (waiting
lists are generally longer in universal health care systems), and, finally, about the current health
care systems found in other industrialized countries (Britain, Canada, France, and Germany all
have universal health care systems). Like in the first study, the knowledge questions were aimed
to be balanced, and their order was randomized for each subject.
Here too, the control group was simply asked these questions in a straightforward, conventional way, with the health care question preceding the domain-specific knowledge questions.
The control group has 247 respondents.
The first treatment group (ST-FLAG) received the questions in the reverse order and was
required to stop and think for 30 seconds before answering the health care question, just like the
ST-FLAG subjects in Brazil. The ST-FLAG treatment group has 251 respondents.
The second treatment group (FACTS, for providing facts about health care) first received ineligibility is actually eligible,” (American Association for Public Opinion Research 2004). Note that RR3 could not be
measured in Brazil because Market Analysis Brazil could not estimate what proportion of cases of unknown eligibility
was actually eligible.
20
The funding awarded for this second study only allowed, unfortunately, for reproducing two of the five treatment groups.
19
formation about health care in the U.S. and then was asked the health care question without
the stop-and-think prompt, just as in the control group. But note the difference between FACTS
and the ANSWERS treatment in Brazil. As in the ANSWERS treatment, the respondents here
are provided information. In the ANSWERS treatment, the information is the correct answers
to the knowledge questions, provided just after each has been asked and answered. In FACTS,
the information was a list of five facts, randomly ordered, constituting the correct answers to the
knowledge questions being asked of the other groups. The FACTS respondents are not asked the
questions, just given the facts constituting the correct answers. The facts were displayed on a
one-page screen with no need to scroll.
Information was provided differently in the U.S. study to prevent ill-informed respondents
from getting discouraged by discovering that they have been answering incorrectly.21 The downside of simply presenting the information is that domain-specific knowledge cannot be measured
for respondents in the FACTS treatment. The FACTS treatment group has 262 respondents.
Finally, note that Knowledge Networks also collects information on socio-demographic variables including age and education. The appendix provides the question wording for all groups.
The expected effects are the same: the stimuli to thought should affect the cognitive processes
people go through when developing an attitude and reduce, in turn, the probability to express
a nonopinion response, especially for the less knowledgeable subjects. The increased thought
conditions, however, may also induced the most knowledgeable to become more uncertain or
ambivalent about their attitude on health care, increasing their probability to express a nonopinion response. But, because of the distribution of political knowledge, thought should, overall,
reduce the expression of nonopinions.
Explaining Nonopinion Responses on Health Care
Table 3 presents the percentage of nonopinion responses on health care for each group. Nonopinions are defined only as those subjects who answered ‘don’t know’ to the health care question
because the questionnaire was self-administered (and thus avoided the pressure to provide the
interviewers with answers to the questions) and also because the health care question explicitly
21
Note that the Brazil study was conducted first and after talking with some of the interviewers, I realized how
rapport could, at times, be affected by providing the right answer after each knowledge question.
20
offered a ‘don’t know’ option. In addition, the question label does not define here the middle
category as the status quo.22
From Table 3, we find that subjects in both treatment groups expressed, as expected, fewer
nonopinions (8.8% in ST-FLAG and 11.4% in FACTS versus 13.4% in the control group), but that
difference was only significant at the conventional .05 (one-tailed) for those in ST-FLAG.
[Table 3 about here.]
But, as argued earlier, nonopinion responses are also affected by other factors and the effects
of thought should be conditioned by knowledge. I present here a model for explaining nonopinion responses on health care that follows that presented for nonopinions on government spending in Brazil. The model defines the probability to express a nonopinion response as a function
of thought, knowledge, age, and education. The equation to estimate is as follows:
P (Yi = 1) = f (β0 + β1 ∗ Agei + β2 ∗ Educationi + β3 ∗ Ki + β4 ∗ Ki2
+β5 STi F LAGi + β6 Ki STi F LAGi + β7
∗
∗
∗
∗
∗
∗
(2)
Ki2 ∗ STi ∗ F LAGi )
where f (X) = 1/(1 + e−X ). The βs are coefficients to be estimated and the subscript i identifies
the individuals.23
The dependent variable here takes on the value of 1 for nonopinion responses as defined
as those who answered ‘don’t know’ to the health care question and 0 otherwise. Knowledge
is measured, again, by summing the number of correct answers to the knowledge questions.
Correct answers were given a value of 1 and incorrect ones, ‘don’t knows,’ and refusals a 0.
Note that the knowledge scale ranges here from 0 to 8 because respondents could score up to 4
points on the last knowledge question by identifying correctly all the countries that have a public
health care system.24 Knowledge and Knowledge2 are also interacted with ST ∗ F LAG to allow
22
Note that adding the middle category to the nonopinion responses, like done in Brazil, does not alter much
the substance of the findings presented here. The effects of thought on nonopinion responses are similar but fail to
reach statistical significance.
23
Subjects in FACTS are excluded from the following analyses because, as explained above, knowledge was not
measured for that group. Note, however, that subjects in that group expressed, on average, fewer nonopinion
responses than those in the control group ( 11.4% vs. 13.4%). Note that this treatment was included in the surveyexperiment to test for other thought-related hypotheses not presented here.
24
To the knowledge questions, 20.5% of the subjects got them all wrong, 24.2% answered one correctly, 23.8%
answered two correctly, 13.9% answered three correctly, 10.2% answered four correctly, 4.4% answered five correctly,
1.7% answered six correctly, 1.3% answered seven correctly, and none got them all right.
21
for thought’s effects to be conditioned by knowledge.25
Table 4 presents the maximum likelihood estimates of a logit model. The results are interesting in many respects. First, as found elsewhere, older and more educated people tend to be
less likely to express nonopinion responses. For example, people aged 65 years old were .11 less
likely to express a nonopinion response as compared to those aged 18. Similarly, people with a
college degree were also .11 less likely to voice a nonopinion as compared with those who have
not completed high school.
[Table 4 about here.]
But more importantly for present purposes, the results reported in Table 4 suggest, once
again, a role for thought, conditioned by knowledge, on explaining nonopinion responses. To
visualize better the role of thought and knowledge on nonopinions, Figure 2 presents the probability of expressing a nonopinion response by knowledge and treatment. For control group
subjects, knowledge first has essentially no effect on the probability of expressing a nonopinion
response, then decreases it as knowledge increases further up to a point where the probability
to express a nonopinion is close to zero. For ST-FLAG subjects, knowledge first reduces substantially the probability of a nonopinion, then increases it as knowledge increases further, just
like in FLAG, ANSWERS, and ST-ANSWERS in Brazil. As a result, thought, again, reduces the
probability of expressing a nonopinion response for the less knowledgeable, but also increases it
for the most knowledgeable.
[Figure 2 about here.]
Once again, subjects who knew basically nothing about health care in the United States (those
who scored 0 on the knowledge scale) did not benefit from the extra thought, just as expected.
Those slightly more knowledgeable (scores of 1 and 2), however, benefited from thinking more
carefully as their probability to express a nonopinion was .11 and .08 lower than those in the
control group, respectively. Both differences are significant at .05 (one-tailed). Thought had essentially no effect on the more knowledgeable subjects who scored 3, 4, and 5. All had low
25
ST and F LAG are the same variable here because the experimental design did not include, this time, a pure ST
or FLAG treatment group.
22
probability of expressing a nonopinion, irrespective of treatment. Finally, the most knowledgeable (scores of 6 and 7) were .08 and .19 more likely to express a nonopinion, respectively, but
these two differences fail to reach statistical significance (due to the small number of subjects
found in these two knowledge groups).
But, in the end, what is the overall effect of thought on nonopinion responses? By calculatating the overall effect of thought, as done in the precedent section, we find that thought reduced
the probability to express a nonopinion response by .04. This effect too is substantial in size, as
it reduced the probability to express a nonopinion by nearly a third, from .13 to .09.
Interestingly enough, the effects of thought in the U.S. are very similar to those reported
in Brazil (smaller in size, obviously, because nonopinions here are only defined as those who
answered ‘don’t know’ to the health care question), by reducing nonopinion responses for the
less knowledgeable, increasing them for the most knowledgeable, but reducing them overall.
Discussion
The results presented in the precedent sections suggest an important role for thought on nonopinion responses. Recall that thought has been stimulated in many different ways: (1) by encouraging subjects to think more carefully; (2) by flagging relevant considerations worth thinking
about; (3) by providing domain-specific information; and (4) by doing a combination of (1) and
(2) or of (1), (2), and (3). Some stimuli produced stronger effects than others, but all experimental treatments reduced nonopinion responses for the less knowledgeable and all, but one
(ST), increased it for the most knowledgeable. But, because of the skewed distribution of political knowledge, thought, overall, reduced substantially the probability to express a nonopinion.
These findings are robust not only because they have been reproduced under different thought
conditions, but also because they have been uncovered in two studies conducted in two culturally different countries (Brazil and the United States), using two distinct interviewing modes
(by telephone in Brazil and over the Web in the United States), and tapping two different issues
(government spending in Brazil and health care in the U.S.).
The effect of thought on the less knowledgeable in reducing nonopinion responses is best
explained by its effect on attitude development. As argued earlier, thought promotes attitude
23
development most likely because it improves comprehension of the attitude-object, helps in the
retrieval of relevant considerations, and facilitates the integration of the retrieved considerations
into an attitude. The reduction in nonopinion responses for that group, therefore, should be
interpreted as the empirical manifestation of attitude development. To be sure, the less knowledgeable do not think much about politics in their everyday lives and, not surprisingly, many of
them do not have attitudes on most political issues. But if given the opportunity to think (or, in
other words, induced to devote more cognitive efforts in processing information) about some of
those issues, many of them rise to the occasion and develop an attitude.
Thought also helps the most knowledgeable in the retrieval of considerations but its effect on
nonopinion responses is different. The most knowledgeable generally think a great deal about
politics and most of them, if not all of them, already have attitudes on most political issues. The
extra thought, however, makes some of them become more uncertain or ambivalent because it
increases their likelihood of retrieving orthogonal or inconsistent considerations with their initial
attitude. This is particularly true when subjects are also provided with information or flagged
with considerations worth thinking about.
But some may argue that the reduction in nonopinion responses observed in the experiments
may simply be an artifact of a different phenomenon. Indeed, the thought stimuli may have
encouraged some people to “manufacture” an answer—one entirely made up to satisfy the interviewer. Testing whether these attitudes are “real” or simply nonattitudes, to use Converse’s
term, is not an easy task but analyses conducted in another paper using the same data show that
thought helps people express attitudes that are more reflective of their underlying interests in
ways similar to providing them with domain-specific information (Gilens 2001).
Now, recall that the theory proposes that thought improves comprehension of the attitudeobject, helps in the retrieval of considerations, and facilitates their integration into an attitude,
but no evidence in support of these assumptions is presented in the paper. Unfortunately, the
ways in which thought is induced in the experiments do not allow to test for them directly. But
the assumption that thought helps in the retrieval of considerations is supported by findings
reported in Zaller and Feldman (1992) where the authors asked a first half of respondents in a
split-sample experiment to voice their thoughts about three issues before expressing where they
stand on those issues and a second half to voice those thoughts after answering the three attitude
24
questions. Again, the first treatment is not a pure “stop-and-think” treatment for reasons discussed earlier, but it is a close cousin. The authors find that respondents induced to think before
answering the attitude questions expressed, on average, 3.7 thoughts per question as compared
to only 2.9 for those respondents that did not have the opportunity to think before answering
the same questions (i.e., those that were required to voice their thoughts after answering them).
Because more considerations were voiced in the first treatment, it is quite plausible that thought
assists in the retrieval of considerations.
The results presented in this paper carry important lessons and implications. First, they provide useful insights into survey, and more specifically questionnaire design. The constant preoccupation for lowering nonopinion responses has led scholars to propose all sorts of question
format and wording, including, most notably, the removal of ‘don’t know’ options (Krosnick
2002). This project has shown that offering opportunities for thought, be it by allowing respondents more time to respond or by asking first questions eliciting considerations worth thinking
about, also lowers the expression of nonopinion responses. This reduction in nonopinion responses benefits social scientists that use survey data because more accurate measures of the
characteristics of populations of interest mean greater validity for their theories on human nature and behavior. In that respect, a count of scientific articles and research notes published in
the last ten years in political science’s top journals reveal that one third of them used some kind
of survey data.26 This speaks to the importance of survey data in our discipline.
More normatively important, however, the results from this project imply that some of the
representational distortions, introduced in polls by the greater tendency of less knowledgeable people to express nonopinion responses, can be palliated simply by stimulating greater
thought.27 To be sure, it is easier to induce people to think more carefully about what they already know than to make them learn. This implication is particularly important for anyone
interested in improving electoral democracy because of the role public opinion plays in politics.
Indeed, citizens are constantly being asked about their political views and behaviors, and their
26
The count includes articles and research notes published in the American Political Science Review (18.8%), the
American Journal of Political Science (32.4%), the Journal of Politics (40.0%), the British Journal of Political Science (37.7%),
and in Political Research Quarterly (41.5%).
27
Thought reduces the expression of nonopinion responses for the less knowledgeable but increases it for the
most knowledgeable. The latter effect should not be worrisome because, as mentioned in an earlier footnote, it
concerns only a very small group of people that generally possess political attitudes on most issues. Moreover, these
people simply need to resolve on the conflicting considerations they have retrieved. The less knowledgeable, on the
other hand, need the extra thought to come to terms with the issues at hand.
25
aggregated answers are reported and discussed heavily in the media. The information polls provide influence the actions of elected officials, and, thus, the kinds of policies and laws that are
enacted. Public opinion, therefore, cannot simply be ignored. The results presented here suggest
ways to make public opinion more representative, by helping those that generally remain silent
to also sing with the choir.
Finally, and more generally, the present paper improves our understanding of how people
come to develop attitudes. The study of attitude development, just like that of attitude change,
is important because of the role political attitudes play in explaining other important political
phenomena like the decision to vote and vote choices, to name a few. The role of knowledge
in explaining why some people hold attitudes while others do not is undeniable, but the results
presented here suggest that thought also plays an important function, by affecting the cognitive
processes people go through when developing an attitude. Moreover, the effects of thought
appear to be more universal than culturally dependent, an important finding for the nascent
field of comparative political psychology.
26
Appendix : Questionnaires
A. Brazil Questionnaire
Question on general government spending
1. Simple version:
Some people think governments (federal, state, and municipal) should reduce taxes, even if it
means providing fewer services. Other people feel it is important for the government to provide
more services even if it means an increase in taxes. Of course, other people think nothing should
be changed in either services or taxes. Do you think governments should increase, decrease, or
make no change in either spending or taxes?
Follow-up question for those subjects who said increase or decrease:
Do you think taxes and services should increase/decrease a little, somewhat or a lot?
2. Stop-and-think version:
Many people spend time thinking about the various aspects of tough or serious choices before
making a decision. Here, we are interested in your views on government spending, which are
very important to us. Thus we’d like to ask you to spend at least thirty seconds thinking about
the question. I will indicate when the thirty seconds have elapsed.
Some people think governments (federal, state, and municipal) should reduce taxes, even if it
means providing fewer services. Other people feel it is important for the government to provide
more services even if it means an increase in taxes. Of course, other people think nothing should
be changed in either services or taxes. Do you think governments should increase, decrease, or
make no change in either spending or taxes?
Please do not respond before the 30 seconds have elapsed. I will tell you when the 30 seconds
have elapsed.
After the 30 seconds have elapsed:
Like I already said, some people think governments (federal, state, and municipal) should
increase taxes and services, others that they should reduce taxes and services, and still others
that nothing should be changed in either services or taxes. What about you? Do you think
governments should increase, decrease, or make no change in either spending or taxes?
Follow-up question for those subjects who said increase or decrease:
Do you think taxes and services should increase/decrease a little, somewhat or a lot?
Knowledge questions (questions were randomly ordered)
Next are some questions about governments in general. We want to see how much information gets out to the public from television, newspapers, and other sources. If you don’t know the
answer, don’t worry about it. Just tell me that you don’t know, and we will move on to the next
question.
1. On which of the following purposes governments in Brazil spent the MOST in 2003?
a
b
c
d
f
National defense
Social security
Culture
Health
Don’t know
Answer: (b) Among the items listed above, governments in Brazil spent the most on social security in
2003.
2. On which of the following purposes governments in Brazil spent the LEAST in 2003?
a National defense
27
b
c
d
f
Social security
Culture
Health
Don’t know
Answer: (c) Among the items listed above, governments in Brazil spent the least on culture in 2003.
3. Which of the following was the federal government’s largest source of revenue in 2003?
a
b
c
d
f
Individual income taxes
Corporate income taxes
Taxes on alcoholic beverages
Taxes on foreign products (imported goods)
Don’t know
Answer: (b) In 2003, the federal government’s largest source of revenue came from the corporate income
taxes.
4. Has Brazil’s consolidated public debt (federal, state, and municipal) increased, decreased, or
stayed about the same since the beginning of the year?
a
b
c
f
Decreased
Stayed about the same
Increased
Don’t know
Answer: (a) Brazil’s consolidated public debt has decreased since the beginning of the year.
5. Is the recent social security reform passed by Lula’s government likely to increase, decrease,
or leave about the same the growth in social security spending in the future?
a
b
c
f
Decreased
Stayed about the same
Increased
Don’t know
Answer: (a) The recent social security reform is likely to reduce the costs of the program in the future.
B. United States Questionnaire
Question on health care
1. Simple version:
There is much concern about the rapid rise in medical and hospital costs. Some people feel
there should be a federal government insurance plan which should cover all medical and hospital expenses for everyone regardless of age, income, and race. Suppose these people are at one
end of a seven-point scale, at point 1. Others feel that all medical expenses should be paid by
individuals, and through private insurance plans or other company paid plans. Suppose these
people are at the other end of the scale, at point 7. People who are exactly midway between are
at point 4. Of course, other people have opinions at points 2 or 3 or 5 or 6. Where would you
place yourself on this scale, or don’t you have an opinion about this? (Seven-point scale shown to
subjects)
2. Stop-and-think version:
Many people spend time thinking about the various aspects of tough or serious choices before
making a decision. Here, we are interested in your views on the role played by the federal
28
government in health care, which are very important to us. We’d like to ask you to spend at least
30 seconds thinking about the question before you answer it. A clock on the upper right side of
the screen will indicate when the 30 seconds have elapsed.
There is much concern about the rapid rise in medical and hospital costs. Some people feel
there should be a federal government insurance plan which should cover all medical and hospital expenses for everyone regardless of age, income, and race. Suppose these people are at one
end of a seven-point scale, at point 1. Others feel that all medical expenses should be paid by
individuals, and through private insurance plans or other company paid plans. Suppose these
people are at the other end of the scale, at point 7. People who are exactly midway between are at
point 4. Of course, other people have opinions at points 2 or 3 or 5 or 6. Where would you place
yourself on this scale, or don’t you have an opinion about this? (Seven-point scale shown to subjects)
Again, this question is very important to us. Please take the next 30 seconds or so to think
carefully about what you think the federal government role in health care should be and how
what it does or doesn’t do may affect your interests, those of your family, or those of the country
as a whole.
Follow-up question for those subjects who said increase or decrease:
Do you think taxes and services should increase/decrease a little, somewhat or a lot?
Knowledge questions (questions were randomly ordered)
Here are some questions about the federal government and health care in general. We want
to see how much information gets out to the public about these topics. If you don’t know the answer, don’t worry about it. Just indicate that you don’t know, and move on to the next question.
1. Would you say the percentage of Americans WITHOUT health care insurance(whether public
or private) is roughly...?
a
b
c
d
f
5%
15%
25%
35%
Don’t know
Answer: (b) The percentage of Americans without health care insurance (whether public or private) is
roughly 15%.
2. Would you say the percentage of the federal budget currently devoted to health care services
like Medicaid and Medicare is roughly...?
a
b
c
d
f
6%
14%
22%
30%
Don’t know
Answer: (c) The percentage of the federal budget currently devoted to health care services like Medicaid
and Medicare is roughly 22%.
3. Next is a series of statements about the costs versus benefits of a more extensive public health
care system covering all medical and hospital expenses for everyone. Please indicate about each
statement whether it’s true, it’s false, or you don’t know whether it’s true or false.
a Those with an average to above average income would receive more than their share of the
benefits compared to what they would pay
29
b Those with a below average income would receive more than their share of the benefits
compared to what they would pay
c Everybody would receive about the same share of the benefits compared to what they
would pay
f Don’t know
Answer: (b) Those with a below average income would receive more than their share of the benefits compared to what they would pay.
4. As compared to private health care systems, do you think waiting lists tend to be:
a Shorter in public health care systems covering all medical and hospital expenses for everyone
b Longer in public health care systems covering all medical and hospital expenses for everyone
c About the same in public health care systems covering all medical and hospital expenses
for everyone and private health care systems
f Don’t know
Answer: (b) Longer in public health care systems covering all medical and hospital expenses for everyone.
5. Which of the following countries have a public insurance plan covering all medical and hospital expenses for everyone? Select all that applies.
a
b
c
d
f
Canada
France
Britain
Germany
Don’t know
Answer: (a, b, c, and d) All the countries listed above have a public insurance plan covering all medical
and hospital expenses for everyone.
References
Althaus, S. L. 1996. “Opinion Polls, Information Effects and Political Equality: Exploring Ideological Biases in Collective Opinion.” Political Communication 13:3–21.
Althaus, S. L. 2003. Collective Preferences in Democratic Politics: Opinion Surveys and the Will of the
People. Cambridge, NY: Cambridge University Press.
Alvarez, R. M. and C. H. Franklin. 1994. “Uncertainty and political Perceptions.” Journal of Politics 56:671–689.
Alvarez, R. M. and J. Brehm. 1995. “American Ambivalence Towards Abortion Policy: Development of a Heteroskedastic Probit Model of Competing Values.” American Journal of Political
Science 39:1055–1082.
Alwin, D. F. and J. A. Krosnick. 1991. “The Reliability of Survey Attitude Measurement: The
Influence of Question and Respondent Attributes.” Sociological Methods and Research 20:139–
181.
30
American Association for Public Opinion Research, AAPOR. 2004. Standard and Definitions: Final
Dispositions of Case Codes and Outcome Rates.
Amir, Y. and I. Sharon. 1987. “Are Social-Psychological Laws Cross-Culturally Valid?” Journal of
Cross-Cultural Psychology 18:383–470.
Ascher, H. 2001. Polling and the Public: What Every Citizen Should Know. Washington, DC: CQ
Press.
Barker, D. C. and S. B. Hansen. 2005. “All Things Considered: Systematic Cognitive Processing
and Electoral Decision-Making.” Journal of Politics 67:319–344.
Bassili, J. N. and J. Fletcher. 1991. “Response-Time Measurement in Survey Research: A Method
for CATI and a New Look at Non-Attitudes.” Public Opinion Quarterly 55:331–346.
Berinsky, A. J. 1999. “The Two Faces of Public Opinion.” American Journal of Political Science
43:1209–1230.
Berinsky, A. J. 2004. Silent Voices: Opinion Polls and Political Representation in America. Princeton,
NJ: Princeton University Press.
Bishop, G. F., A. J. Tuchfarber and R. W. Oldendick. 1986. “Opinions on Fictious Issues: The
Pressure to Answer Survey Questions.” Public Opinion Quarterly 50:240–250.
Bohner, G., G. Moskowitz and S. Chaiken. 1995. “The Interplay of Heuristic and Systematic
Processing of Social Information.” European Review of Social Psychology 6:33–68.
Chaiken, S., A. Liberman and A. H. Eagly. 1989. “Heuristic and Systematic Information Processing Within and Beyond that Persuasion Context.” In Unintended Thought, ed. J. S. Uleman
and J. A. Bargh. New York, NY: Guilford Press.
Chen, S. and S. Chaiken. 1999. “The Heuristic-Systematic Model in Its Broader Context.” In DualProcess Theories in Social Psychology, ed. S. Chaiken and Y. Trope. New York, NY: Guilford
Press.
Conover, P. J. and D. D. Searing. 2002. “Expanding the Envelope: Citizenship, Contextual
Methodologies, and Comparative Political Psychology.” In Thinking About Political Psychology, ed. J. H. Kuklinski. New York, NY: Cambridge University Press.
Converse, J. M. 1987a. Survey Research in the United States: Roots and Emergence 1890-1960. Berkeley, CA: University of California Press.
Converse, P. E. 1987b. “Changing Conceptions of Public Opinion in the Political Process.” Public
Opinion Quarterly 51:S12–S24.
Converse, P. E. and R. Pierce. 1986. Political Representation in France. Belknap Press.
Eagly, A. H. and S. Chaiken. 1993. The Psychology of Attitudes. Forth Worth: TX: Harcourt Brace
Jovanovich.
Ehrlich, H. J. 1964. “Instrument Error and the Study of Prejudice.” Social Forces 43:197–206.
Faulkenburry, G. D. and R. Mason. 1978. “Characteristics of Nonopinionation and No Opinion
Response Groups.” Public Opinion Quarterly 42:533–543.
31
Fazio, R. H. 1990. “Multiple Processes by Which Attitudes Guide Behavior: The MODE Model
as na Integrative Framework.” In Advances in Experimental Social Psychology, Vol. 23, ed. L.
Berkowitz. New York, NY: Academic Press.
Fishhoff, B. 1991. “Value Elicitation: Is There Anything There?” American Psychologist 46:835–847.
Flanders, J. P. and D. L. Thistlethwaite. 1967. “Effect of Familiatization and Group Discussioon
upon Risk-Taking.” Journal of Personality and Social Psychology 5:91–95.
Fournier, P. and M. Turgeon. unpublished. “Survey-Induced Deliberation: Evidence of a New
Question Order Effect.” Manuscript, University of North Texas .
Francis, J. and L. Busch. 1975. “What We Know About ’I Don’t Knows’.” Public Opinion Quarterly
39:207–218.
Gallup, G. and S. Rae. 1940. The Pulse of Democracy: The Public Opinion Poll and How it Works.
New York: NY: Greenwood Press.
Geer, J. G. 1996. From Tea Leaves to Opinion Polls: A Theory of Leadership. New York: NY: Columbia
University Press.
Gilens, M. 2001. “Political Ignorance and Collective Policy Preferences.” American Political Science
Review 95:379–396.
Gosnell, H. F. 1940. “The Polls and Other Mechanisms of Democracy.” Public Opinion Quarterly
4:224–228.
Green, D. P. and A. S. Gerber. 2002. “Reclaiming the Experimental Tradition in Political Science.”
In Political Science: The State of the Discipline, ed. I. Katznelson and H. V. Milner. New York:
NY: W. W. Norton and Company.
Groves, R. 1989. Survey Errors and Survey Costs. New York: NY: John Willey and Sons.
Jacobs, L. R. and R. Y. Shapiro. 2000. Politicians Don’t Pander: Political Manipulation and the Loss of
Democratic Responsiveness. Chicago: IL: University of Chicago Press.
Kinder, D. R. and T. R. Palfrey. 1993. “On Behalf of an Experimental Political Science.” In Experimental Foundations of Political Science, ed. D. R. Kinder and T. R. Palfrey. Ann Arbor: MI:
University of Michigan Press.
King, G., M. Tomz and J. Wittenberg. 2000. “Making the Most of Statistical Analyses: Improving
Interpretation and Presentation.” American Journal of Political Science 44:347–361.
Krosnick, J. A. 1991. “Response Strategies for Coping with the Cognitive Demands of Attitude
Measure in Surveys.” Applied Cognitive Psychology 5:213–236.
Krosnick, J. A. 2002. “The Causes of No-Opinion Responses to Attitude Measures in Surveys:
They Are Rarely What They Appear to Be.” In Survey Nonresponse, ed. R. M. Groves, D. A.
Dillman, J. L. Eltinge and R. J. A. Littles. New York: NY: John Wiley and Sons.
Krosnick, J. A. and M. A. Milburn. 1990. “Psychological Determinants of Political Opinionation.”
Social Cognition 8:49–72.
Linville, P. W. 1982. “The Complexity Extremity Effect and Age Based Stereotyping.” Journal of
Personality and Social Psychology 42:193–210.
32
McDermott, R. 2002. “Experimental Methods in Political Science.” Annual Review of Political
Science 5:31–61.
McGuire, K. T. and J. A. Stimson. 2004. “The Least Branch Revisited: New Evidence on Supreme
Court Responsiveness to Public Preferences.” Journal of Politics 66:1018–1035.
Millar, M. G. and A. Tesser. 1986. “Thought-Induced Attitude Change: The Effects of Schema
Structure and Commitment.” Journal of Personality and Social Psychology 51:259–263.
Mishler, W. and R. S. Sheehan. 1993. “The Supreme Court as a Countermajoritarian Institution?
The Impact of Public Opinion on Supreme Court Decisions.” American Political Science Review 87:87–101.
Moghaddam, F. M., D. M. Taylor and S. C. Wright. 1992. Social Psychology in Cross-Cultural Perspective. New York: NY: W. H. Freeman.
Nisbett, R. E. 2003. The Geography of Thought: How Asians and Westerners Think Differently... and
Why. New York: NY: The Free Press.
Petty, R. E. and D. T. Wegener. 1998. “Attitude Chnnge: Multiple Roles for Persuasion Variables.”
In The Handbook of Social Psychology, ed. D. Gilbert, S. T. Fiske and G. Lindzey. New York:
NY: McGraw-Hill.
Petty, R. E. and J. T. Cacioppo. 1986a. Communication and Persuasion: Central and Peripheral Routes
to Attitude Change. New York: NY: Springler Verlag.
Petty, R. E. and J. T. Cacioppo. 1986b. “The Elaboration Likelihood Model of Persuasion.” Advances in Experimental Social Psychology 19:124–203.
Quirk, P. J. and J. Hinchliffe. 1998. “The Rising Hegemony of Mass Opinion.” Journal of Policy
History 10:19–50.
Sadler, O. and A. Tesser. 1973. “Some Effects of Salience and Time Upon Interpersonal Hostility
and Attraction.” Sociometry 26:99–112.
Sanbonmatsu, D. M. and R. H. Fazio. 1990. “The Role of Attitudes in Memory-Based Decision
Making.” Journal of Personality and Social Psychology 59:614–622.
Schuman, H. and S. Presser. 1981. Questions and Answers in Attitude Surveys: Experiments on
Question Form, Wording and Content. New York: NY: Academic Press.
Schwartz, N. and S. Sudman. 1992. Context Effects in Social and Psychological Research. New York:
NY: Springler-Verlag.
Sigelman, L. 1981. “Question-Order Effects on Presidential Popularity.” Public Opinion Quarterly
45:199–207.
Stimson, J. A., M. B. MacKuen and R. S. Erickson. 1995. “Dynamic Representation.” American
Political Science Review 89:543–565.
Tesser, A. 1976. “Thought and Reality Constraints as Determinants of Attitude Polarization.”
Journal of Research in Personality 10:183–194.
Tesser, A. 1978. “Self-Generated Attitude Change.” In Advances in Experimental Social Psychology,
Vol. 11, ed. L. Berkowitz. New York: NY: Academic Press.
33
Tesser, A. and C. Cowan. 1975. “Some Effects of Thought and Number of Cognitions on Attitude
Change.” Social Behavior and Personality 3:165–173.
Tesser, A. and C. Leone. 1977. “Cognitive Schemas and Thought as Determinants of Attitude
Change.” Journal of Experimental Social Psychology 13:340–356.
Tesser, A. and M. C. Conlee. 1975. “Some Effects of Time and Thought on Attitude Polarization.”
Journal of Personality and Social Psychology 31:262–270.
Tetlock, P. E. 1993. “Cognitive Structural Analysis of Political Rhetoric: Methodological and
Theoretical Issues.” In Explorations in Political Psychology, ed. S. Iyengar and W. McGuire.
Durham: NC: Duke University Press.
Tomz, M., J. Wittenberg and G. King. 2003. CLARIFY: Software for Interpreting and Presenting
Statistical Results. Version 2.1. In http://gking.harvard.edu/.
Tourangeau, R., K. Rasinski and R. D’Andrade. 1991. “Attitude Structure and Belief Accessibility.” Journal of Experimental Social Psychology 27:48–75.
Tourangeau, R., L. J. Rips and K. Rasinski. 2000. The Psychology of Survey Response. New York:
NY: Cambridge University Press.
Verba, S. 1996. “The Citizen as Respondent: Sample Surveys and American Democracy.” American Political Science Review 90:1–7.
Wilson, T. D., D. Kraft and D. S. Dunn. 1989. “The Disruptive Effects of Explaining Attitudes:
The Moderating Effect of Knowledge About the Attitude Object.” Journal of Experimental
Social Psychology 25:379–400.
Wilson, T. D., D. S. Dunn and D. Lisle. 1989. “The Disruptive Effects of Explaining Why We Feel
the Way We Do.” In Advances in Social Psychology, ed. L. Berkowitz. Orlando: FL: Academic
Press.
Wilson, T. D., D. S. Sunn, J. A. Bybee, D. B. Hyman and J. A. Rotondo. 1984. “Effects of Analyzing
Reasons on Attitude-Behavior Consistency.” Journal of Personality and Social Psychology 47:5–
16.
Wright, J. R. and R. G. Niemi. 1983. “Perceptions of Issue Positions.” Political Behavior 5:209–223.
Zaller, J. 1992. The Nature and Origins of Mass Opinion. New York: NY: Cambridge University
Press.
Zaller, J. and S. Feldman. 1992. “A Simple Theory of the Survey Response: Answering Questions
Versus Revealing Preferences.” American Journal of Political Science 36:579–616.
34
Table 1: Percentage of Nonopinion Responses on Government Spending, Brazil
35
Nonpionions
N
Total
35.0
1055
Control
41.8
181
Pooled treatment
33.5*
874
ST
33.8
175
FLAG
32.7*
176
ANSWERS ST-FLAG
33.3*
33.2*
176
175
ST-ANSWERS
34.7
172
Note: Entries are percentages. Nonopinions are defined here as the sum of those respondents who ventured a “Don’t know” response or
placed themselves at the status quo (No change in either spending or taxes) on the government spending 7-point scale.
*Indicates statistically significant differences between treatment and control at the conventional .05 level or higher (one-tailed).
Table 2: Explaining Nonopinion Responses on Government Spending, Brazil
Variables
Knowledge
Knowledge2
Control
ST
FLAG
.67*
(.40)
-.33*
(.13)
.04
(.39)
-.09
(.11)
-.63†
(.40)
.13
(.11)
-.23
(.38)
-.49
(.38)
-.39
(.35)
-.52†
(.37)
.12†
(.10)
-.00
(.00)
.02
(.03)
-.43
(.36)
Age
Education
Constant
Coefficients
(s.e.)
ANSWERS ST-FLAG
Pseudo-R2
Log-likelihood
N
ST-ANSWERS
.05
(.42)
.02†
(.12)
-.38†
(.37)
.07†
(.10)
-.89*
(.40)
-.39
(.37)
.03
-662.34
1049
*Indicates statistically significant coefficients at .10 or higher.
†Indicates treatment group coefficients that are statistically different from those in the control
group at .10 or higher.
36
Table 3: Percentage of Nonopinion Responses on Health Care, U.S.
Nonopinions
N
Total
11.2
760
Control
13.4
247
Pooled treatment
10.2
513
ST-FLAG
8.8*
251
FACTS
11.4
262
Note: Entries are percentages. Nonopinions here are simply defined as those who answered
“Don’t know” to the health care question.
*Indicates statistically significant differences between treatment and control at the conventional
.05 level or higher (one-tailed).
37
Table 4: Explaining Nonopinion Responses on Health Care, U.S.
Variables
Knowledge
Knowledge2
Age
Education
Constant
Coefficients
(s.e.)
Control ST-FLAG
.19
(.54)
-.26
(.19)
-1.19*†
(.44)
.15†
(.09)
-.02*
(.01)
-.17*
(.10)
.29
.25
(.64)
(.65)
Pseudo-R2
Log-likelihood
N
.15
-147.94
498
*Indicates statistically significant coefficients at .10 or higher.
†Indicates treatment group coefficients that are statistically different from those in the control
group at .10 or higher.
38
Predicted Probability
Figure 1: Predicted Probability of Expressing a Nonopinion Response on Government Spending
by Knowledge and Treatment Group, Brazil
1. ST
.7
.6
.5
.4
.3
.2
.1
0
2. FLAG
3. ANSWERS
0
4. ST−FLAG
.7
.6
.5
.4
.3
.2
.1
0
0
1
2
3
1
5. ST−ANSWERS
4
5 0
1
2
3
4
5
knowledge
Control
Treatment
Graphs by Treatment Group
Note: The bars report 90% confidence intervals.
39
90% CI
90% CI
2
3
4
5
Figure 2: Predicted Probability of Expressing a Nonopinion Response on Government Spending
by Knowledge and Treatment Group, U.S.
Predicted Probability
.4
.3
.2
.1
0
0
1
2
3
4
Knowledge
Control
ST−FLAG
Note: The bars report 90% confidence intervals.
40
5
90% CI
90% CI
6
7