GEORGE F. BISHOP, ROBERT W. OLDENDICK,
AND ALFRED J. TUCHFARBER
IN CONSTRUCTING questions about public affairs, researchers must
often decide whether to present respondents with one or both sides of
an issue. On the question of gun control, for instance, an ABC
News/Washington Post poll (1981) asked respondents simply whether
they favored or opposed "stronger legislation controlling the distribution of handguns." One could, however, have asked them a more
"balancecT two-sided question such as the following from split-ballot
experiments by Schuman and Presser (1978) on this issue:
Would you favor a law which would require a person to obtain a police
permit before he could buy a gun, or do you think such a law would interfere
too much with the right of citizens to own guns.
Such a choice between question forms can be crucial, for it has
frequently been found that offering respondents a second substantive
Abstract Data from a split-ballot experiment show that offering respondents an alternative position on an issue not only affects the marginals, it also influences whether
respondents will give an opinion at all. Furthermore, the study demonstrates that these
form effects can occur despite the use of filter questions which theoretically screen out
those who tend to be most susceptible to such effects: the less educated or uninformed.
The analysis does provide evidence, however, that less educated respondents are
indeed more affected by differences in question format and that they are much more
likely to "acquiesce" to one-sided agree/disagree forms. In discussing the results the
authors develop an information-processing model of question form effects and a methodological strategy to generate further research on a much-needed theory of the survey
instrument.
George F. Bishop is a Senior Research Associate, Robert W. Oldendick is Assistant
Director, and Alfred J. Tuchfarber is Director of the Behavioral Sciences Laboratory at
the University of Cincinnati's Institute for Policy Research. The research reported here
was supported by a grant from the National Science Foundation (SOC78-07407). The
authors want to thank Howard Schuman and Stanley Presser for their comments on an
earlier version of this paper.
PuNk Opinion Quarterly Vol. 46:69-85 © 1982 by The Trustee-, of Columbii University
Pubfiihed by ENevier Nortb-Hoiland, Inc.
0033-362X/K2/0O46-69rt2.5O
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Effects of Presenting
One Versus Two Sides of an
Issue in Survey Questions
70
BISHOP, OLDENDICK, AND TUCHFARBEH
Research Design
As part of a larger project on question form effects we designed a
field experiment in which respondents were randomly assigned to one
of two filter questions and to either a one-sided presentation of an
issue in agree/disagree format or to a two-sided presentation in
forced-choice form. This created a simple two-by-two factorial design
through which we tested the joint effects of differently worded filters
and one- versus two-sided formats on responses to five public affairs
issues (see Appendix). All the issue questions represent
simulations—that is, not exact replications—of items which have ap-
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
alternative on an issue will significantly decrease the percentage of
respondents endorsing the first alternative from a single-sided form
(Hedges, 1979; Kalton et al., 1978; Noelle-Neumann, 1970; Payne,
1951; Rugg and Cantril, 1944; Schuman and Presser, 1977, 1978). One
might, therefore, draw quite different conclusions about the distribution of public opinion on a particular issue when using one question
form rather than another.
Here we extend the research on effects of presenting one versus
two sides of an issue in several ways. First, we look at the influence
of these question forms in combination with two types of filter questions. The use of such screening devices should presumably reduce, if
not eliminate, the differences in response between one- and two-sided
formats since they remove the less informed respondents, who tend to
be most susceptible to form effects (see, e.g., Bishop et al., 1980).
Second, we assess the impact of one- versus two-sided forms not only
upon the marginal distribution of substantive responses, but also upon
the more basic matter of whether respondents will give opinions at all.
Third, we test a hypothesis derived from previous work by Schuman
and Presser (1977) that less educated respondents are more readily
affected by agree/disagree versus forced-choice formats, including
their well-known susceptibility to the acquiescence response set (see,
e.g., Jackman, 1973). In discussing the results of our experiments we
also address the practitioner's question about whether one should
avoid one-sided presentations of an issue because of the potential
"biasing" effects, as well as the more fundamental problem of
whether—in presenting both sides of an issue—we are dealing
basically with differences in the wording of the same issue, or rather,
different issues. Finally, we develop an information-processing model
of question form effects which not only helps explain some of the
findings reported here, but also provides a methodological strategy for
expanding this line of research.
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
71
peared in the Michigan SRC/CPS election studies. One of the randomized subgroups in our experiment, for example, received the
following version of a single-sided item from the pre-1964 SRC election surveys:1
A second subgroup was administered a slight variation of a two-sided
form of this issue which was first introduced in the SRC 1964 election
study, in combination with a completely new filter question:2
Some people feel that the government in Washington should see to it that
every person has a job and a good standard of living. Others think the
government should just let each person get ahead on his own. Have you been
interested enough in this to favor one side over the other? [Italics added]
(IF YES) DO you think that the government should see to it that every
person has a job and a good standard of living, or should it let each person get
ahead on his own?
A third subgroup was exposed to the same single-sided form as the
first group, but in combination with the filter used in the second
group, that is, "Have you been interested enough in this to favor one
side over the other?" The fourth subgroup was given the two-sided
form administered to the second group, but with the filter used in the
first group—that is, "Do you have an opinion on this or not?" (see
1
The original version of this item, which appeared in the election studies between
1956 and 1960, read as follows: "The government in Washington ought to see to it that
everybody who wants to work can find a job." [Italics added] The filter question used
then was also worded in a slightly different way—"Now would you have an opinion on
this or not?" [Italics added] In addition, it was asked in a five-point, Likert type of
response format, with categories on the interviewer's hand card ranging from "Agree
strongly; government definitely should" to "Disagree strongly; government definitely
should not" (see The SRC 1956 American National Election Study, revised ICPR
edition, 1974). Thus our dichotomous and somewhat differently worded version of this
and other items from that period represent but approximate "simulations" of those
forms which we have designed primarily to assess the effects of one- versus two-sided
presentations of an issue. As with any such experimental simulation, the results can be
generalized to the original setting only by theoretical extension. Note here too that the
SRC election studies were all conducted by personal interviews, whereas our experiments were all done by telephone interviews—another limiting condition.
2
In the SRC 1964 survey, this item was prefaced by the words, "In general."
Otherwise, our versions of this and other questions from that study were quite similar.
For the exact wordings, see The SRC 1964 American National Election Study, revised
ICPR edition, 1974. The exact wording and format for the women's equality issue can
be found in the American National Election Study, 1978, Volume II (2nd ICPSR
edition, 1979).
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Some people feel that the government in Washington should see to it that
every person has a job and a good standard of living. Do you have an opinion
on this or not? [Italics added]
(IF YES) Do you agree or disagree with the idea that the government should
see to it that every person has a job and a good standard of living?
72
BISHOP, OLDENMCK, AND TUCHFARBER
Findings
EFFECTS OF QUESTION FORMS AND FILTERS ON OPINION GIVING
First, we consider a much neglected question in the literature: Does
offering respondents a second substantive alternative on an issue
Table 1. Abbreviated Forms of the Question About the School Desegregation Issue*
Form A: "Do youagree on disagree with the idea that the government should stay out of
the question of whether white and black children go to the same schools?"
Form B: "Do you think the government should see to it that white and black children go
to the same schools, or stay out of this area as it is none of its business?"
Form C: "Do you agree or disagree with the idea that the government should see to it
that white and black children go to the same schools?"
* For the complete wording and format of these items, as well as their associated
filters, see the Appendix.
3
This study was conducted in Hamilton County, Ohio, which includes the City of
Cincinnati. The response rate (i.e., completed interviews) for this survey was 73.6
percent (N = 1,218); the refusal rate, 15.7 percent. The remainder consisted of partially
completed interviews (3.3. percent), potential respondents unable to complete the
interview because of a language barrier, a hearing problem, or severe illness, etc. (4.7
percent), and those who could not be contacted during the field period, e.g., people
away on business or vacation (1.9 percent).
4
The response rate here was 74.6 percent; the refusal rate, 10.7 percent. The rest
were distributed in a pattern similar to that in the first survey.
5
Treating time of the survey as a variable—i.e., Nov.-Dec., 1978 versus May-June,
1979—we found no significant interaction with time in the response by format by filter
relations shown in Tables 2 and 3 below. That is, the effects of format and filtering on
opinion giving and on the direction of the substantive responses to the various issues
were essentially of the same magnitude in each survey. Thus we collapsed the two
experiments into one pooled data file.
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Appendix). These latter two conditions thus constitute experimental
variations of our own and are designed to counterbalance the first
two, factorially.
Table 1 shows one other wrinkle in the design: three versions of the
school desegregation issue. Form A represents a single-sided version
of an SRC question from the early election studies (1956-1960). Form
C offers a "positively" worded reversal of Form A, which we constructed primarily to evaluate acquiescence response effects. Form B,
on the other hand, provides a two-sided variation of Form C which
makes comparisons with the other four issues possible (see also the
Appendix).
We gathered the data for our first experiment with these question
forms through an omnibus RDD telephone survey of citizens (18 and
over) in a major U.S. metropolitan area.3 Six months later (MayJune, 1979) we replicated the experiment within the same omnibus
vehicle.4 Since the results of the two experiments were quite similar,
we have pooled them in the analyses summarized below.5
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
73
Table 2. Percentage of Respondents Giving an Opinion on a Public Affairs Issue by Type
of Response Format and Type of Filter Question
"Do You Have an Opinion
on 1Ttis or Notr
Agree/
Public Affairs Issue 1Disagree
Power of the
federal government
Government guaranteed employment
Fair treatment of
blacks in jobs &
housing
Equal role for women
in business,
industry &
government
Government involvement in school
desegregation:
Form C vs. B
Government involvement in school
desegregation:
Form A
Subst.
Choice
Diff.'
"Havt• You Been Interested
Enough in This to Favor
One iSide over the Other?"
Agree/
Disagree
Choice
Diff.'
xn
63.8
(566)
80.2
(566)
69.0
(588)
83.1
(587)
5.2**
1.43
59.7
(588)
83.5
(588)
69.9
(554)
84.5
(554)
10.2*
82.7
(585)
86.2
(551)
3.5
82.2
(566)
86.0
(587)
93.1
(553)
7.1*
83.2
(285)
85.5
(553)
86.4
(301)
1.0
Subst.
2.9
.31
82.9
(584)
.7
.91
87.3
(566)
89.7
(585)
2.4
2.3
—
—
—
—
—
—
—
—
—
3.91*
• All the statistical significance levels indicated in this column are based on a simple
chi-squared test of the relation between response format and opinion giving (df = 1).
b
These are likelihood ratio chi-squared values computed with the ECTA program
developed by Leo Goodman and his associates at the University of Chicago; they
represent a test of whether the relation between response format and opinion" giving is
significantly stronger with one type of filter than the other (df for each test = 1).
*p < .05.
**p < .10.
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
increase their inclination even to give an opinion? Stating a counterargument, for example, may, simply because it provides a respondent more time to "think about" the topic, lead the respondent to
verbalize a thought. Or, one may argue that the additional cues
contained in the second alternative evoke different cognitions about
the issue which somehow induce respondents into expressing an
opinion. Whatever the process, we found that on two of the five
issues, presenting another substantive choice indeed stimulated significantly higher percentages of respondents to give an opinion (Table
2). Though this form effect also seems to have occurred more often
with one version of the filter than the other—that is, "Do you have an
opinion on this or not?'—the interaction of opinion giving and response format with type of filter reached statistical significance only
on the women's equality issue. Notice too that the tendency to give an
74
BISHOP, OLDENDICK, AND TUCHFARBEH
EFFECTS OF QUESTION FORMS AND FILTERS ON THE MARGINALS
The figures in Table 3 show, as expected, that adding another
substantive choice to a question makes a significant difference in the
marginals. In fact, for eight of the nine available comparisons, offering a second substantive alternative decreased endorsement of the
initial single-sided statement. And on five of these eight the difference
reached statistical significance.
Furthermore, these findings demonstrate that such format effects
can occur despite the use of filter questions which theoretically screen
out those who tend to be most susceptible to them (see, e.g., Bishop
et al., 1978). Nor do we find any evidence in Table 3 that the
influence of question format depends upon the type of filter, indicating once more that the two filters were essentially equivalent (see,
again, Table 2). In the remaining analyses we therefore collapse the
two filter conditions.
Surprisingly, perhaps, the substantive alternative that "women's
place is in the home" made little difference in the distribution of
responses to the women's equality issue. So while this conventional
counterargument may have "provoked" more opinion giving (see
Table 2), it evidently could not offset the contemporary movement
toward the idea of equal rights for women. We say "provoked" be-
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
opinion did not vary significantly with type of filter, suggesting that
they were psychologically equivalent. The filtering manipulation, in
other words, did not produce a significant "main effect."
No obvious clues come to mind as to why the second substantive
alternative on just two of these issues should have stimulated more
opinion giving than those on other issues. It does not appear to be due
to differences in the nature of the extra choice (see Appendix). On the
issue of governmental power, for instance, the second alternative
represented a logical antithesis of the first: "Do you think the government is getting too powerful, or do you think the government has
not gotten too strong?" [Italics added] Whereas with the question on
women's equality in the workplace, the second option symbolized a
qualitatively distinct assertion which is not necessarily antithetical
and which may indeed have elicited a rather different cluster of
cognitions, namely, the statement that "women's place is in the
home." Yet it generated an effect on opinion giving which, while
statistically significant, was not as large as that created by the more
direct, but token-like reversal of the first alternative on the issue of
governmental power. These puzzling results, along with others reported below, underscore the need for a theory of the cognitive
processes underlying such response effects.
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
75
Table 3. Percentage of Respondents Agreeing with a One-Skied Statement in
Agree/Disagree Format vs. Percentage Selecting the Same Statement over a Second
Alternative in a Two-Sided Substantive Choice Format, by Type of Filter Question
"Do You Have an Opinion
on 1fhis or NotT'
Agree/
Public Affairs Issue Disagree
72.4
(351)
44.0
(491)
66.9
(387)
30.3
(468)
62.6
(484)
54.1
(475)
83.6
(505)
85.2
(515)
36.3
(237)
27.5
(473)
74.2
(260)
—
Agree/
Disagree
Subst.
Choice
75.9
(361)
40.5
(454)
Diff.'
x2b
63.3
(406)
32.4
(488)
12.6*
2.30
8.1*
1.52
57.6
(465)
54.8
(484)
2.8
1.58
86.2
(494)
84.8
(525)
1.4
1.11
8.8**
—
—
—
—
—
—
—
—
—
Diff.'
5.5
13.7*
8.5*
-1.6
* See note a in Table 2.
" Likelihood ratio chi-squared values for a test of whether the relation between format
and substantive response is significantly stronger with one type of filter than the other (df
for each test = 1).
*p < .01
** p < .05.
cause merely mentioning the idea that a woman's place is in the
home may have irritated some respondents to speak up and reject it
by choosing the more (socially desirable) egalitarian alternative. Offering respondents another choice on an issue, in other words, will
not necessarily attract them, though it may make them more likely to
"think about" expressing an opinion.
Another consequential finding in Table 3 is the difference in percentage of agreement between the two one-sided versions of the
school desegregation issue (see Table 1). Since forms A and C were
worded in the opposite direction—that is, the government should
"stay out" versus "see to it"—the acquiescence effect would be 36.3
percent - (100.0 percent - 74.2 percent) = 36.3 percent - 25.8
percent = 10.5 percent. A simple chi-squared test indicated that this
form effect was likewise statistically significant (p < .05, df = 1).
Although an effect of 10 percent on the marginals may not seem
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Power of the
federal government
Government guaranteed employment
Fair treatment of
blacks in jobs &
housing
Equal role for women
in business,
industry &
government
Government involvement in school
desegregation:
Form C vs. B
Government involvement in school
desegregation:
Form A
Subst.
Choice
"Have You Been Interested
Enough in This to Favor
One Side over the Other?"
7*
BISHOP, OLDEND1CK, AND TUCHFARBEK
unusually large, it could become crucial in a policy-oriented survey
where the population splits closer to 50/50, leading one to infer that
the majority of respondents are either "integrationists" or "segregationists." Question wording thus can have substantive consequences (see Roshco, 1978).
INTERACTION EFFECTS OF EDUCATION
(a) Among those with at least some college education, the acquiescence
effect was 34.5 percent - (100.0 percent - 67.5 percent) = 34.5 percent
- 32.5 percent = 2.0 percent.
(b) among those with a high school education, it was 30.1 percent - (100.0
percent — 76.8 percent = 30.1 percent — 23.2 percent = 6.9 percent.
(c) Among those with less them a high school education, it was 50.0 percent
- (100.0 percent - 75.4 percent) = 50.0 percent - 24.6 percent = 25.4
percent.
Not surprisingly, then, the likelihood ratio chi-squared value for this
interaction effect was statistically significant (x1 = 6.53, df = 2, p <
.05).6 These results thus reinforce those of earlier studies which have
found less educated respondents to be much more likely to "acquiesce" (see, e.g., Jackman, 1973). Indeed, such respondents may be
prone to form effects of all kinds (see Bishop et ah, 1980, 1981),
leading one to make rather different inferences about their opinions,
depending upon the question form or context.
* This and the other likelihood ratio chi-squared values presented in this paper were
calculated with the computer program ECTA, developed by Leo Goodman and his
associates at the University of Chicago for performing significance tests in multivariate
contingency tables (see Davis, 1974).
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Previous work by Schuman and Presser (1977) suggests that less
educated respondents tend to be most affected by shifts from agree/
disagree to forced-choice formats, although they also found that the
appearance of such interactions depended upon the nature of the
issue. Foreign policy items, for instance, created weaker effects than
other issues in their experiments. The percentages in Table 4 tell
much the same story. For there we find that the differences between
the two formats were generally larger among the less educated
groups, especially among those with less than a high school education. But only on one of the five issues—government guaranteed
employment—did the apparent interaction of the question form effect
with education approach statistical significance.
In contrast, the differences by level of education between the two
one-sided forms of the school desegregation question (see, again, the
wording of Forms A and C in Table 1) look much more striking:
30.1
(93)
6.4
—
—
28.1
(423)
—
—
34.5
(87)
67.5
(311)
76.8
(250)
86.0
(357)
.9
93.0
(454)
92.1
(419)
56.7
(337)
4.2
58.4
(423)
3.3
62.6
(388)
72.2
(248)
43.7
(327)
68.6
(379)
25.2
(412)
7.6*
Diff'
Agree/
Disagree
76.2
(332)
28.5
(393)
Subsl.
Choice
—
—
26.8
(354)
85.2
(364)
49.3
(347)
63.7
(278)
28.7
(345)
Subst.
Choice
High School
61.2
—
—
75.4
(171)
50.0
(56)
3.3
(219)
70.0
(217)
•
71.9
(128)
64.8
(216)
.8
7.4'*
15.0*
8.5*
Diff.'
—
—
36.2
(207)
68.2
(214)
56.8
(183)
57.5
(127)
49.7
(191)
Subsl.
Choice
—
—
13.8**
—
—
1.05
.44
.38
4.4
1.8
5.52**
.72
14.4*
15.1*
X2"
Diff.'
Less than High School
Agree/
Disagree
• See note a in Table I.
b
Likelihood ratio chi-squared values for a test of whether the relation between format and substantive response is significantly different among
educational groups (df for each test •= 2).
• p < .05.
** p < .10.
Power of the
federal government
Government guaranteed
employment
Fair treatment of
blacks in jobs
& housing
Equal role for women
in business, industry
& government
Government involvement
in school desegregation:
Form C vs. B
Government involvement
in school desegregation:
Form A
Public Affairs Issue
Agree/
Disagree
Some College +
Table 4. Percentage of Respondents Agreeing with a One-Sided Statement in Agree/Disagree Format vs. Percentage Selecting the Same Statement over
a Second Alternative in a Two-Sided Substantive Choice Format, by Level of Education
poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
o
s
§
2
>
z
o
1
1
7»
BISHOP, OLDENDICK, AND TUCHFARBER
Discussion
Consider what happens, cognitively, when a respondent is asked to
agreeor disagree with a particular statement of opinion—for example,
"that the government in Washington is getting too powerful for the
good of the country and the individual person." Other things being
equal (e.g., political partisanship), if the respondent has given little or
no thought to this topic, then he is unlikely to have a counterargument
to oppose the statement. Thus he will be inclined to "go along" and
agree with it, to acquiesce. We hypothesize that it is the better
educated respondents who are most likely to have such counterarguments or qualifying cognitions (such as "it depends") about an issue
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
This experiment demonstrates, once again, that survey results on
issues of public affairs can be markedly influenced by variations in
question wording and format. But repeatedly demonstrating such effects, without any model of the processes underlying them, will result
in little progress on the problems of survey measurement and,
perhaps, saturation with this sort of work (see Schuman and Presser,
1977). We would therefore like to propose a conceptual framework
that not only helps make sense of the form effects reported here, but
also gives some direction for developing a more general theory of
"response effects" in surveys (cf. Bradburn, forthcoming).
Our approach stems largely from recent advances in cognitive social psychology. Numerous studies, for example, have shown that
verbal reports of various kinds tend to be based heavily on the
information that is most available to respondents in memory (see,
e.g., Ericsson and Simon, 1980; Nisbett and Ross, 1980; Wyer and
Carlston, 1979). Putting this proposition in the context of the survey
interview, we postulate that when respondents are asked a question
about their opinions on a particular issue, they do not perform an
exhaustive search for representative instances of that opinion in
long-term memory; instead, they respond "with the first thing that
comes to mind," or more formally, in terms of the information that is
most accessible to them in memory, either from the immediate context of the interview schedule or from the environmental context of
the recent past. As a corollary, we would expect that less educated
respondents would be most likely to give such top-of-the-head reactions to survey questions since they have little other public affairs
information to draw upon. In other words, the more information one
has available in memory about a given topic (e.g., politics), the less
susceptible one will tend to be to differences in the information that is
immediately available in alternative wordings of a question on that
topic.
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
19
7
Note, however, that simply because a two-sided form acts as an "equalizer" it is not
necessarily a belter form. A glance back at Table 4, for instance, will show that the
association between education and opinions on the government guaranteed employment
issue tended to be significantly stronger with the agree/disagree than with the
substantive-choice form (x2 = 5.52, p = .063).
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
available to them in memory, since they are generally more informed
about public affairs (see Hyman, 1975); hence, they are more likely to
disagree with all kinds of opinion statements, as we know from a long
line of social research (see Jackman, 1973; Lenski and Leggett, 1960).
Indeed, it is this unavailability of alternative cognitions or counterarguments which underlies much of the classic acquiescence response
style in surveys and which has a well-known association with level of
education—as we found, for example, in our analysis of the acquiescence effect on the school desegregation issue in Table 4.
This same principle also explains what happens with the forcedchoice form which provides respondents a second substantive alternative: it makes a counterargument available to them. And as we
know from the findings in Table 4, less educated respondents were
most likely to be influenced by the availability of the second alternative, the one, that is, which was most immediately recent in shortterm memory at the time of answering the question. In other words,
by making a counterargument available to less educated respondents
with the forced-choice form, we give them the cognitive "equalizer"
that is normally more available to their better educated counterparts
in the face of a typical agree/disagree item.7 The important point here
is not whether some respondents are giving more "liberal" reactions
on the agree/disagree version of an issue because of an acquiescence
response set as opposed to more "conservative" reactions on the
forced-choice form simply because they are provided with another
view; they are, in fact, doing both of these things—which differ
superficially—but doing so for the same underlying reason: namely,
answering the question in terms of whatever information is most
accessible to them in memory.
How, though, would we account for the variation in form effects
among the specific issues in this experiment and their differential
interactions with education? Notice that the two issues in Table 4
which deviated from the general pattern of greater form effects with
decreasing levels of education—women's equality in the workplace
and fair treatment for blacks in jobs and housing—were each characterized by relatively high levels of "agreement" in the single-sided
form. This means that they constituted strong arguments, appealing
as they did to respondents' basic sense of "fair play," and were
therefore difficult to argue against. The alternative that "women's
80
BISHOP, OLDENDICK, AND TUCHFARBER
• Such random probing should also help illuminate why some items stimulate more
opinion-giving than others (Table 2), by revealing the types of cognitions that are
evoked by offering respondents either one or two alternatives on an issue. More
generally, probing respondents to "think aloud" about their reactions to items provides
a useful, but neglected tool for understanding the cognitive processes which underlie
the expression of opinions in the survey interview.
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
place is in the home," for instance, evidently represented a rather
weak counterargument against women's equality in the workplace. So
too did the statement that it "is not the federal government's business"
to see that black people get fair treatment in jobs and housing make
for a feeble reply, relatively speaking. On the other hand, the statement that "the government in Washington is getting too powerful for
the good of the country and the individual person" also elicited substantial agreement. Yet the counterargument—"others feel that the
government in Washington has not gotten too strong"—proved sufficiently persuasive to reduce acceptance of that proposition, particularly among those with less than a high school education (see, again,
Table 4).
Although admittedly ad hoc, this line of interpretation suggests a
basic strategy for generating predictions about the effects of presenting one versus two sides of an issue in survey questions—that is, in
terms of the strength of arguments and counterarguments. Some
statements of opinion, in other words, are simply more compelling
and difficult to argue against than others. Conversely, not all counterarguments in a two-sided form are equally effective on a given
issue. Other things being equal, then (for example, a respondent's
prior level of information), it might be useful to think of a respondent's probability of choosing a specific alternative in a two-sided
form as a function of the relative strength of each argument (that is,
the greater the strength of an available counterargument, the greater
will be the response effect). In this Way we can look at the survey
interview as a microcosmic communication and persuasion experiment, in which the interviewer is presenting one- versus two-sided
statements that are more or less persuasive communications which
the respondent is asked to accept or reject. Therefore, the task becomes one of assessing the strength of alternative positions on an
issue. Having a group of "experts" on the issue make such judgments
would be one way to proceed, although this is often fraught with
problems of rater biases. Alternatively, one might begin by randomly
probing respondents in a pilot study (cf. Schuman, 1966) about the
"reasons" why they agree or disagree with a given statement of
opinion and, thereby, produce a list of arguments and counterarguments which could be pitted against one another, systematically, in a
follow-up study.8 Questions about the relative strength of arguments
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
M
Conclusion
For one reason or another most of us have fallen into question
construction habits, the substantive consequences of which we are
hardly aware of. The preference for favor or oppose, approve or
disapprove, and agree or disagree formats are but a few examples. As
we have demonstrated here, such choices can significantly affect not
only the marginal distributions on a given issue, but also whether a
respondent will even venture an opinion on the issue. But contrary to
the impression the results may have given, the solution does not lie in
making sure that our items present more than one side of an issue.
For as we have seen, it is no easy matter to decide what the "other
side" of the issue should be. In some cases what seems to be a simple
logical reversal on an issue (e.g., "the government has not gotten too
strong") creates a significant form effect, whereas a more radical
counterargument (e.g., "women's place is in the home") has hardly
any effect at all. Furthermore, we have found that such form effects
depend on a respondent's level of education. So that it becomes
questionable whether we should say that one format or another is
"biased" or "unbalanced" since that is clearly a relative matter. Biased
for whom? Only those with less than a high school education? Or
does that depend upon the issue? For that matter, should we even
bother to think about whether there is a "true" or "correct" wording
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
and counterarguments on an issue could thus be answered empirically, and items constructed on a firmer basis than habit or intuition.
Had such knowledge been available about the SRC/CPS items used in
the present experiment, we would have been able to make some fairly
specific predictions about their response effects. Now, at least, we
have some idea on how to proceed in the future.
This theoretical exposition should suffice to communicate the utility
of an information-processing approach to the study of question form
effects in the survey interview. We have elsewhere extended this
model, with a fair degree of success, to the analysis of question order
and context effects (Bishop et al., 1980). The next step will be to test
the propositions we have advanced here by directly manipulating the
nature and strength of the cognitions or counterarguments that are
available to respondents just prior to administering them a particular
question form. In this way, for example, we can test whether the
acquiescence phenomenon can be systematically increased or decreased. If so, we would be able to demonstrate that acquiescence is
not so much a respondent "trait" as it is a function of the information
that is made available to respondents in a specific context.
82
BISHOP, OLDENDICK, AND TUCHFARBER
and format for a given issue? Or are there only more or less useful
ones for a given purpose of explanation or prediction?
The solution, in our judgment, lies not so much in methodological
refinements as in theory-building, an effort which we believe will take
the form of an information-processing model of the survey instrument.
FORM W
1. "Now some people are afraid that the government in Washington is getting too
powerful for the good of the country and the individual person. DO YOU HAVE AN
OPINION ON THIS OR NOT?" (IF YES) "Do you agree or disagree with the idea that
the government is getting too powerful for the good of the country and the individual
person?'
2. "Some people feel that the government in Washington should see to it that every
person has a job and a good standard of living. DO YOU HAVE AN OPINION ON
THIS OR NOT?' (IF YES) "Do you agree or disagree with the idea that the government should see to it that every person has a job and a good standard of living?"
3. "Some people say that if black people are not getting fair treatment in jobs and
housing, the government should see to it that they do. DO YOU HAVE AN OPINION
ON THIS OR NOT?" (IF YES) "Do you agree or disagree with the idea that if black
people are not getting fair treatment in jobs and housing, the government should see to
it that they do?"
4. "Some people feel that women should have an equal role with men in running
business, industry, and government. DO YOU HAVE AN OPINION ON THIS OR
NOT?" (IF YES) "Do you agree or disagree with the idea that women should have an
equal role with men in running, business, industry, and government?"
5. "Some people say that the government in Washington should stay out of the
question of whether white and black children go to the same schools. DO YOU HAVE
AN OPINION ON THIS OR NOT?" (IF YES) "Do you agree or disagree with the idea
that the government should stay out of the question of whether white and black
children go to the same schools?"
FORM x
'
1. "Now some people are afraid that the government in Washington is getting too
powerful for the good of the country and the individual person. Others feel that the
government in Washington has not gotten too strong for the good of the country and the
individual person. HAVE YOU BEEN INTERESTED ENOUGH IN THIS TO
FAVOR ONE SIDE OVER THE OTHER?" (IF YES) "What is your feeling—do you
think the government is getting too powerful, or do you think the government has not
gotten too strong?"
2. "Some people feel that the government in Washington should see to it that every
person has a job and a good standard of living. Others think the government should just
let each person get ahead on his own. HAVE YOU BEEN INTERESTED ENOUGH
IN THIS TO FAVOR ONE SIDE OVER THE OTHER?" (IF YES) "What is your
feeling—do you think the government should see to it that every person has a job and a
good standard of living, or should it let each person get ahead on his own?"
3. "Some people feel that if black people are not getting fair treatment in jobs and
housing, the government should see to it that they do. Others feel that this is not the
federal government's business. HAVE YOU BEEN INTERESTED ENOUGH IN
THIS TO FAVOR ONE SIDE OVER THE OTHER?." (IF YES) "How do you
feel—should the government in Washington see to it that black people get fair treatment
in jobs and housing, or do you feel that this is not the federal government's business?"
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Appendix: Issue Question Formats and Filters
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
«3
FORM Y
1. "Now some people are afraid that the government in Washington is getting too
powerful for the good of the country and the individual person. HAVE YOU BEEN
INTERESTED ENOUGH IN THIS TO FAVOR ONE SIDE OVER THE OTHERr
(IF YES) "Do you agree or disagree with the idea that the government is getting too
powerful for the good of the country and the individual personr
2. "Some people feel that the government in Washington should see to it that every
person has a job and a good standard of living. HAVE YOU BEEN INTERESTED
ENOUGH IN THIS TO FAVOR ONE SIDE OVER THE OTHERr (IF YES) "Do
you agree or disagree with the idea that the government should see to it that every
person has a job and a good standard of livingr
3. "Some people feel that if black people are not getting fair treatment in jobs and
housing, the government should see to it that they do. HAVE YOU BEEN INTERESTED ENOUGH IN THIS TO FAVOR ONE SIDE OVER THE OTHERr (IF
YES) "Do you agree or disagree with the idea that if black people are not getting fair
treatment in jobs and housing, the government should see to it that they d o r
4. "Some people feel that women should have an equal role with men in running
business, industry, and government. HAVE YOU BEEN INTERESTED ENOUGH
IN THIS TO FAVOR ONE SIDE OVER THE OTHERr (IF YES) "Do you agree or
disagree with the idea that women should have an equal role with men in running
business, industry, and governmentr
5. "Some people say that the government in Washington should stay out of the
question of whether white and black children go to the same schools?' HAVE YOU
BEEN INTERESTED ENOUGH IN THIS TO FAVOR ONE SIDE OVER THE
OTHERr (IF YES) "Do you agree or disagree with the idea that the government
should stay out of the question of whether white and black children go to the same
schoolsr
FORM Z
1. "Now some people are afraid that the government in Washington is getting to
powerful for the good of the country and the individual person. Others feel that the
government in Washington has not gotten too strong for the good of the country and the
individual person. DO YOU HAVE AN OPINION ON THIS OR N O T r (IF YES)
"What is your feeling—do you think the government is getting too powerful, or do you
think the government has not gotten too strongr
2. "Some people feel that the government in Washington should see to it that every
person has a job and a good standard of living. Others think the government should just
let each person get ahead on his own. DO YOU HAVE AN OPINION ON THIS OR
NOTr (IF YES) "What is your feeling—do you think the government should see to it
that every person has a job and a good standard of living, or should it let each person
get ahead on his ownf
3. "Some people feel that if black people are not getting fair treatment in jobs and
housing, the government should see to it that they do. Others feel that this is not the
federal government's business. DO YOU HAVE AN OPINION ON THIS OR NOTT
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
4. "Some people feel that women should have an equal role with men in running
business, industry, and government. Others feel that women's place is in the home.
HAVE YOU BEEN INTERESTED ENOUGH IN THIS TO FAVOR ONE SIDE
OVER THE OTHERr (IF YES) "What do you think—do you feel that women should
have an equal role with men in running business, industry and government, or do you
feel that women's place is in the home?"
5. "Some people say that the government in Washington should see to it that white
and black children go to the same schools. Others claim that this is not the government's
business. HAVE YOU BEEN INTERESTED ENOUGH IN THIS TO FAVOR ONE
SIDE OVER THE OTHERr (IF YES) "Do you think the government in Washington
should see to it that white and black children go to the same schools, or stay out of this
area as it is none of its businessr •
M
BISHOP, OLDENWCK, AND TUCHFARBEH
FORM
ww
[Questions 1 to 4 were identical to those in Form W (above).]
5. "Some people say that the government in Washington should see to it that white
and black children go to the same schools. DO YOU HAVE AN OPINION ON THIS
OR NOTT (IF YES) "Do you agree or disagree with the idea that the government
should see to it that white and black children go to the same schools?"
References
ABC News/Washington Post Poll
1981 "Support for handgun control grows after Reagan shooting . . ." April, Appendix C.
Bishop, George F., Robert W. Oldendick, and Alfred J. Tuchfarber
1978 "Effects of question wording and format on political attitude consistency."
Public Opinion Quarterly 42:81-92.
1980 "Experiments in filtering political opinions." Political Behavior 2:339-70.
1981 "Question order and context effects in measuring political interest." Paper
presented at the annual conference of the American Association for Public
Opinion Research, Buck Hill Falls, Pa., May 1981.
Bradbum, Norman M.
forth- "Response effects." In P. H. Rossi and J. D. Wright (eds.), The Handbook of
coming Survey Research. New York: Academic Press.
Davis, James A.
1974 "Hierarchical models for significance tests in multivariate contingency tables." Pp. 189-231 in H. L. Costner (ed.), Sociological Methodology 19731974. San Francisco: Jossey-Bass.
Ericsson, K. Anders, and Herbert A. Simon
1980 "Verbal reports as data." Psychological Review, 87:215-51.
Hedges, B. M.
1979 "Question wording effects: presenting one or both sides of a case." The
Statistician 28:83-99.
Hyman, Herbert H., Charles R. Wright, and John S. Reed
1975 The Enduring Effects of Education. Chicago: University of Chicago Press.
Jackman, Mary R.
1973 "Education and prejudice or education and response-set7" American
Sociological Review 38:327-39.
Kalton, Graham, Martin Collins, and Lindsay Brook
1978 "Experiments in wording opinion questions." Applied Statistics 27:149-61.
Lenski, Gerhard, and John C. Leggett
1963 "Caste, class, and deference in the research interview." American Journal of
Sociology 65:463-67.
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
(IF YES) "How do you feel—should the government in Washington see to it that black
people get fair treatment in jobs and housing, or do you feel that this is not the federal
government's business?"
4. "Some people feel that women should have an equal role with men in running
business, industry, and government. Others feel that women's place is in the home. DO
YOU HAVE AN OPINION ON THIS OR NOT?" (IF YES) "What do you think—do
you feel that women should have an equal role with men in running business, industry,
and government, or do you feel that women's place is in the home?"
5. "Some people say that the government in Washington should see to it that white
and black children go the the same schools. Others claim that this is not the government's business. DO YOU HAVE AN OPINION ON THIS OR NOT?' (IF YES)
"Do you think the government in Washington should see to it that white and black
children go to the same schools, or stay out of this area as it is none of its business?'
ONE VS. TWO SIDES OF AN ISSUE IN QUESTIONS
«
Downloaded from http://poq.oxfordjournals.org/ at Penn State University (Paterno Lib) on May 18, 2016
Nisbett, Richard, and Lee Ross
1980 Human Inference: Strategies and Shortcomings of Social Judgment. Englewood Cliffs, N.J.: Prentice-Hall.
NoeHe-Neumann, Elisabeth
1970 "Wanted: rules for wording structured questionnaires." Public Opinion
Quarterly 34:191-201.
Payne, Stanley L.
1951 The Art of Asking Questions. Princeton, NJ.: Princeton University Press.
Roshco, Bernard
1978 "The polls: polling on Panama—si; don't know; hell, noT Public Opinion
Quarterly 42:551-62.
Rugg, Donald, and Hadley Cantril
1944 "The wording of questions." Pp. 23-50 in H. Cantril (ed.), Gauging Public
Opinion. Princeton, N.J.: Princeton University Press.
Schuman, Howard
1966 "The random probe: a technique for evaluating the validity of closed questions." American Sociological Review 31:218—22.
Schuman, Howard, and Stanley Presser
1977 "Question wording as an independent variable in survey analysis." Sociological Methods and Research 6:151-70.
1978 "Attitude measurement and the gun control paradox." Public Opinion Quarterly 41:427-38.
Wyer, Robert S., and Donald E. Carlston
1979 Social Cognition, Inference, and Attribution. Hillsdale, N.J.: Lawrence
Erlbaum Associates.