Univer.
Rehabilitating the Raters: An Assessment of
Interest Group Ratings of Congress
r, Balti.
terly 93
R. STEVEN DANIELS
University of Alabama at Birmingham
r, (rev.
n
New
Abstract
This study examines the validity of interest group ratings of Congress during the
period 1959 to 1981. The ratings themselves have been controversial. Many members of
Congress find them to be distorted and unrepresentative. The arguments for and against
ratings are examined in a dimensional context. Most interest group ratings load very
highly on a single liberal-conservative dimension. Contrary to the arguments of rating
critics, both single- and general-issue groups produce ratings which are representative of
the underlying dimension, suggesting that on broad questions of political philosophy,
most ratings are valid measures.
INTRODUCTION
You get two kinds of money in politics. Sometimes people feel that if they can buy you
early theyll be in good shape after you get here. The other is that if they agree with you
theyll try to keep you here, and if they don't theyll try to bounce you out. Today, we
have the situation where there are threats based on individual votes. Ten thousand
dollars may come in against you from one group because of a single vote. . . . Voters
have less and less ability to understand what the overall record is, and what the individual legislator is doing to and for them, because you have all these messages beamed
at them on individual issues. . . . Representative David Obey (D-WI), quoted in Drew
(1983, p. 90)
. . . [Elach of the staff members [of Americans for Democratic Action (ADA), the Committee on Political Education of the AFL-CIO (COPE), and the Chamber of Commerce
of the United States (CCUS)]expressed considerable surprise and some skepticism at the
use to which scholars put their ratings, and none of them attributed much influence to
the scores among the general public (Fowler, 1982: 403).
Evaluating members of Congress o n the basis of their voting records has
been a scholarly activity since first proposed b y Orin Libby (1897). The first interest
group to prepare such evaluations w a s the American Federation of Labor (AFL) in
1906.' The National Farmer's Union followed in 1919 (Keller, 1981; National
hTners Union, 1982). T h e first continuous general-circulati~nanalysis of congressional roll-call voting w a s undertaken by the New Republic magazine in 1938. Many
other groups have followed in the footsteps of these pioneering organizations. BY
November 1984 the total number of organizations which had at some time rated
CONGRESS& THE PRESIDENCY VOLUME 16
NUMBER 1 SPRING 1989
24 1 CONGRESS & THE PRESIDENCY
members of Congress had risen to 92.2Given the selective nature of these ratingsand
the political consequences arising from them, the controversy with which their
publication has been surrounded is not surprising. The purpose of this article is to
assess the validity of the charges of distortion which have been made against interest
group ratings,
A REVIEW AND ANALYSIS OF RATING PROCEDURES
An understanding of the nature of the controversy surrounding the practice of interest group rating requires some knowledge of the mechanisms by which the ratings
are calculated. For most groups, the process of rating consists of five steps. First, the
group selects ten to 40 roll-call votes from the pool of available votes recorded for
the most recent session of Congress. Second, the group determines the direction of
each vote, i.e., whether yea or nay best represents the group's position on that roll
call. Third, the group decides whether to weight certain votes more than others
because of their importance to group interests. Fourth, the group then compares the
votes of each member of Congress on the selected roll calls and calculates the percentage of the time the member agrees with the group's positions. Fifth, the group then
uses the ratings, among other things, as positive and negative publicity for members
with high or low scores, as cues to guide direct lobbying activities, and as criteria to
distribute campaign contributions (Keller, 1981: 511).
Rating critics (including many members of Congress) suggest that these procedures distort the voting records of members of Congress because of the limited sample of votes, the overrepresentation of particular issues, and the selection of issues to
reward friends and punish enemies. The key problem seems to be one of accountability; most detractors find the procedures to be highly arbitrary and unrepresentative of the voting records of members of Congress either in general or in specific
issue areas (Sabato, 1984; Sorauf; 1984-1985). The greatest criticism is reserved for
single-issue groups. To rating critics, their narrow focus, their stridency, and their
unwillingness to compromise make these groups' ratings especially distorting. The
quote by David Obey which opens this article represents a significant pool of opinion in Congress.
The groups themselves consider the ratings a legitimate extension of their lobbying efforts. Congressional roll-caII votes are a matter of public record and a significant step in the legislative process, and the groups feel that they are within their
First Amendment rights to publicize their agreements and disagreements with the
positions taken by individual legislators. Moreover, virtually all groups believe that
their selection processes produce ratings which are an accurate reflection of the policy interests that they are pursuing in Congress (see, e.g., National Farmers Union,
1982; National Federation of Independent Business, 1982; Ripon Society, 1984; and
National Taxpayers Union, 1984).
Single-issue groups also have their defenders. Sylvia Tesh (1984) has argued
that single-issue groups are in the classic tradition of American participator/
P , ~on ~the~issue
~ of~narrow
~ ~focus,
. she has noted that virtually all of the so-ca11ed
single issues are. in practice, highly constrained. Most single-iss~egroups link their
Issue
to a broader liberal or conservative philosophy.3 Moreover, herexPe-
rier
circ
on
mo
I
Pro
the
1
his
19f
gre
to
I
pei 1
clo
cal
sta
vo
fle
ral
mc
I
in1
ar
W(
to
SU
th
rc
tv
tE I
tc
e:
P1
Ti
I€
g
TI
U
h
81
v
r
S
S
I I
1:
REHABILITATING THE RATERS ( 25
,sand
their
is to
terest
ntertings
,the
i for
In of
t roll
hers
s the
centhen
bers
la to
oce-
amato
untnta:ific
for
heir
The
in-
obnifleir
the
hat
101-
311,
md
led
)rY
led
eir
3e-
,ience has suggested that these groups are willing to compromise under the proper
circumstances; however, she also has noted that most single-issue groups are based
on principle rather than economics and that, therefore, compromise may not be the
most acceptable alternative.
The arguments of both proponents and opponents of ratings suggest that the
process by which ratings are generated should have a demonstrable impact on both
the individual ratings and on the relationship among the various ratings. Poole and
his associates (Poole, 1981; Poole and Daniels, 1985; Poole and Rosenthal, 1985,
1986, 1988) have demonstrated the relevance of the spatial model of voting to congessional roll-call behavior and interest group vote selection procedures. According
to their research, if legislators and interest groups can be assumed to have singlepeaked issue preferences and if these actors tend to select issue positions which are
closest to them in the issue space, the issue positions of both members and groups
can be represented in an issue space of low dimensionality. Under such circumstances, an interest group's rating of a legislator will reflect the member's general
voting record on the group's issues. The more that the group's selection of issues reflects the general content of the issue space, the more likely it will be that the group's
ratings will accurately reflect both its position and the position of the individual
member in that underlying consensus.
The techniques used to calculate interest group ratings can introduce distortion
into this issue space in three ways. First, the group can select roll-call votes which
are consistent with its political position, but do not reflect the general issue space
well. In a multi-dimensional analysis, such groups will load poorly on a general factor, but will load highly on specific factors. Any attempt to use the group's ratings to
summarize member's positions in the general space will distort the member's record.
Second, the group's roll-call selection procedures can lead to a poor recovery of
the member's positions in the space. This problem arises if the group chooses too few
roll-call votes on which to base the calculation (some groups have chosen as few as
two votes), or if the group chooses roll-call votes too far from its own position. In
the first case, there may not be enough information about the legislator's preferences
to produce a unique statistical solution. In the second case, considerable error in
estimation is introduced into the analysis. In either case, the recovered rating is a
poor reflection of the member's true position and the statistical fit of the group's
ratings to the results of a multi-dimensional analysis of a large number of groups and
legislators will be low.
The third type of distortion is subtler. Many groups, whether single-issue or
general-issue, discuss their vote selections and calculated ratings in terms of a narrow set of issue interests, That is, most such groups perceive their issue positions as
unique. If these perceptions are literally true, then statistically such groups should
load on multiple unique dimensions while the remaining groups would load on a
@-era1 dimension or a set of general dimensions. The only forms of distortion
would be of the first or second type noted above. However, in practice some narrowly-focused groups tend to select roll-call positions which are controversial or are
selected by more than one group. such groups are very likely to load highly on the
same general dimensions as general-issue groups (Tesh, 1984). AS long as the issues
selected to calculate the ratings genuinely reflect group interests, the resulting member ratings cannot be considered distorted. Distortion occurs if the group claims a
26 1 CONGRESS & THE PRESIDENCY
narrow focus, but selects roll-calls SO as to pursue a general set of ideological goals.
The presence or absence of each of the types of distortion implies a particular
kind of statistical universe. If the single-issue group claims of uniqueness are accurate and distortion largely arises from poor issue selection (type 2) or the use of
single-issue ratings as surrogates for general ideological positions (type I), a spatial
analysisof the ratings of many such groups should produce a multidimensional solution. The first factor should represent the general underlying liberal-conservative
dimension and should be loaded with the ratings of general issue groups. The re]atively narrow interests of most single-issue groups should produce additional dimensions reflecting the diverse policy interests of the groups. Most importantly, the
analysis should be "noisy"; the relationship between the group ratings and the derived dimensions should be relatively low. Single-issue groups should be less wellrepresented in the space than general issue groups. Groups who select too few issues
for their ratings or who select issue positions far from their own will also fit poorly
in the derived issue space.
If, on the other hand, the vote selections of both general-issue and single-issue
groups are derived from a common pool of controversial roll-call votes and distortion is primarily produced by the misrepresentation of general ideological ratings as
narrowly focused issue-based ratings (type 3), spatial analysis of the ratings should
produce a common, evaluative space of low dimensionality. Since the groups are
selecting roll-call votes for their calculations which accurately reflect their positions
on the specific issues (which are closest to them in the issue space), these groups'
ratings should show a very high fit to the recovered dimension or dimensions (Poole
and Rosenthal, 1985). The fit of single-issue groups on the statistical model should
be as high as the fit of groups with a general-issue orientation because single groups
are drawing from this same underlying space (Tesh, 1984). This will be as true of
single-issue groups pursuing an ideological agenda under the guise of a narrow issue
focus as it will of groups whose issue selections are narrowly focused but are drawn
from the common pool. Poor fit to the model will be restricted to groups whose
agendas are truly unique and groups subject to the second type of distortion discussed above. In summary, the debate over rating distortion suggests the following
set of alternate hypotheses:
To
fro
inc
mi
we
Th
ori
Pu
riv I
Lo
de
tio
an
ist I
inl
g*'
19
esl
Pe
Cc
I
PC
10
I
m
I
I
I
Hypothesis 1 - The nonrepresentative nature of the selected votes used to calculate
interest group ratings will produce a set of ratings whose spatial structure will consist of
several dimensions: a general dimension reflecting the underlying unidirnensionality of
the space and several specific dimensions reflecting the ~ o l i c yinterests of the various
groups. The fit of a significant number of khe groups to the dimensions should be d a lively low.
Hypothesis 1A - Single-issue groups should be less likely to load o n the general
dimension and should have lower overall fits with all dimensions.
E(
a1
a1
Hypothesis 2 - The unidimensional character of the congressional roll-calls from which
the groups are selecting will produce a set of ratings whose spatial structure is unidimensional. The fit of most groups to this single dimension should be high.
Hypothesis 2A - Single-issue groups should fit as well as general-issue groups on
this dimension.
T
a1
v,
t€
I
1
REHABILITATING THE RATERS 27
:a1 goals.
larticular
re accurle use of
a spatial
nal solp
Ervative
The rela11dimen~tly,the
1 the deess well'wissues
t poorly
gle-issue
i distor~tingsas
i should
)upsare
ositions
groups'
s (Poole
' should
groups
true of
IW issue
:drawn
; whose
ion disllowing
8
ulate
ist of
ty of
rious
rela-
neral
rhich
nen-
s on
TO test these competing expectations, a spatial analysis of interest group ratings
from the period 1959 to 1981 was p e r f ~ r m e dRatings
.~
by 40 different groups were
included; however, since 16 ratings by Congressional Quarterly, four by the Cornmittee for the Survival of a Free Congress, and two by the Chamber of Commerce
were also added to the data set, the total number of "groups" in the analysis is 59.
The inclusion of the various ratings calculated by Congressional Quarterly, while the
organization is not strictly an interest group in the context of this paper, serves two
purposes. First, the inclusion of the various presidential support scores and the derived liberal-conservative scores (Conservative Coalition, Larger Federal Role, and
Lower Federal Spending) serves as a mechanism for calibrating and identifying the
derived set of dimensions. Second, and equally important, presidential administrations are acting as rating groups when they identify important roll-calls in Congress
and suggest an appropriate vote. Thus, presidential support scores reflect an administration's snapshot of congressional support for the administration's program just as
interest group ratings represent a summary of congressional attitudes toward the
group's legislative agenda.
The years that each group rated are listed in Table 1.During the period 1959 to
1981, 1,258 representatives and 261 senators served in the U.S.Congress. The interest groups produced a total of 203,387 ratings for these lawmakers over a %year
period.
The procedure by which spatial coordinates for the groups and members of
Congress are derived is a generalized form of the least square unfolding algorithm of
Poole (1981, 1984). The ratings are transformed into distances by subtracting from
100 and dividing by 50 (the latter is arbitrary to give the final product an approximate two-unit range).
In [the] scaling model, the legislators are represented in terms of s Euclidean coordinates
k = l , . . .,s, where s is the dimensionality of the space, t indexes time, and i indexes
the legislators. We impose the restriction that at time t, xikt is a polynomial function of
time. That is:
Xikt,
where m is the degree of the polynomial used to approximate the temporal behavior of
the legislators and interest groups. Interest group coordinates are similarly restricted.
(Poole and Rosenthal, 1986: 4).
The unfolding algorithm estimates all time ~eriodssimultaneously. Using
Eckart-Young or singular value decomposition, a space of lower-order dimensionality is derived which minimizes the sum of the squared error between the derived
and actual distance matrices between all groups and all members of Congress.5
'lI~eseprocedures are superior to factor analysis for this data set because factor
analysis performs singular value decomposition on the correlation matrix across
variables rather than on the raw distance matrix. As a result, factor analysis would
fend to overestimate the dimensionality of the space, since moderate groups would
1
28 1 CONGRESS & THE PRESIDENCY
TABLE 1
Description of Interest Groups in the Unfolding Analysis
Abbreviation
ACLU
ACU
ACA
ADA
AFBF
AFGE
COPE
AFSCME
AFT
ASCNSI
BFW
BCTD
CCUS
CCUS2
CWLA
cv
CVVF
SANE
CNFMP
No. of
Years
Years Rating
Group
American Civil Liberties Union (G)
American Conservative Union (G)
Americans for Constitutional Action (G)
Americans for Democratic Action (G)
American Farm Bureau Federation (E)
American Federation of Government Employees
(El
AFL-CIO Committee on Political Education (E)
American Federation of State, County and
Municipal Employees (E)
American Federation of Teachers (E)
American Security Council National Security
Index (S)
Bread for the World (S)
Building and Construction Trades Department
(AFL-CIO) (E)
Chamber of Commerce of the United States (El
CCUS 2nd Rating 1 9 7 9 Senate (NOT)
Child Welfare League of America (S)
Christian Voice (G)
Christian Voters Victory Fund (G)
Citizens for a Sane World (S)
Coalition for a New Foreign and Military Policy
6)
CFSCA
CFSCE
CFSCD
CFSCS
CCAUSE
CW
CC
LFR
LFS
IKE
PFPIKE
PDPIKE
JFK
Committee for the Survival of a Free Congress:
All lssues (G)
Committee for the Survival of a Free Congress:
Economic lssues (NOT)
Committee for the Survival of a Free Congress:
Defense lssues (NOT)
Committee for the Survival of a Free Congress:
Social lssues (NOT)
Common Cause (GI
Congress Watch (GI
Conservative Coalition (Congressional Quarterly
-CQ) (NOT)
Larger Federal Role (CQ) (NOT)
Lower Federal Spending (CQ) (NOT)
Eisenhower Presidential Support Score:
All lssues (CQ) (NOT)
Eisenhower Presidential Support Score:
Foreign Policy (CQ) (NOT)
Eisenhower Presidential Support Score:
Domestic Policy (CQ)
Kennedy Presidential Support Score: All Issues
(CQ) (NOT)
1
REHABILITATING THE RATERS 29
-
3
mployees
tion (El
and
lcurlty
Irtment
States [E)
.y Policy
longress:
iongress:
longress:
longress:
Abbreviation
No. o f
Years
PFPJFK
3
PDPJFK
3
LBJ
5
PFPLBJ
5
PDPLBJ
5
NlXON
FORD
CARTER
REAGAN
CFA
FCNL
LCV
LWV
LL
6
3
NASC
NCSC
NEA
NFO
NFU
NFlB
NTU
NWPC
NEWREP
RIPON
TWR
UAW
UMW
Years Rating
Group
Kennedy Presidential Support Score:
Foreign Policy (CQ) (NOT)
Kennedy Presidential Support Score:
Domestic Policy (CQ) (NOT)
Johnson Presidential Support Score: All Issues
(CQ) (NOT)
Johnson Presidential Support Score:
Foreign Policy (CQ) (NOT)
Johnson Presidential Support Score:
Domestic Policy (CQ) (NOT)
Nixon Presidential Support Score (CQ) (NOT)
Ford Presidential Support Score (CQ) (NOT)
Carter Presidential Support Score (CQ) (NOT)
Reagan Presidential Support Score (CQ) (NOT)
Consumer Federation of America (S)
Friends' Committee on National Legislation (G)
League of Conservation Voters (S)
League of Women Voters (GI
4
1
10
4
10
10
11
Liberty Lobby (G)
National Alliance of Senior Citizens (S)
National Council of Senior Citizens (S)
National Education Association (E)
National Farmers Organization IE)
National Farmers Union (El
National Federation of Independent Business (El
National Taxpayers Union [GI
National Women's Political Caucus IS)
New Republic (NOT)
Ripon Society (G)
Taxation with Representation (G)
United Auto Workers (El
United Mine Workers (E)
4
4
12
8
17
5
10
1
14
11
2
13
2
- -
- - -
Source: Poole and Rosenthal (I
9 8 6 : 4 5 - 4 6 ) , Table 1.
Quarterly
,II Issues
-
Note: The groups above are classified into four categories: general-issue (GI, single-issue ( S ) ~
economic (E), and not classified (NOT).
30 1 CONGRESS & THE PRESIDENCY
Poole and Rosenthal (1986: 8) found that a single dimension with no change
over time accounted for 74 percent of the variation in the original ratings. The accuracy of the single dimensional model could be increased to 76.5 percent by allowing for linear trend. The addition of a second dimension only added an additional
five percent. The unidimensional solution with linear trend appears to describe the
data adequately. This supports work done by Kritzer (1978) on the 91st and 93rd
Houses, by Schneider (1979) on the 92nd through the 94th Congresses, and by
Hoadley (1980) on the 92nd through 95th Congresses. Poole and Rosenthal (1985)
have developed an unfolding algorithm which operates directly on the individual
roll-call votes. Their most recent research (Poole and Rosenthal, 1988) concludes
that roll-call voting from 1789 to 1985 is largely unidimensional, and that from
about 1903 on, this dimension is largely stable.
The dimensional coordinates, linear trend estimates, and r-squares for the
ratings of the 40 interest groups are presented in Table 2. The dimensional coordinates run from negative (liberal) to positive (conservative). Computationally, the
total shift that a group makes over the rating period is twice the linear trend estimate. To make the estimates comparable (since the rating period for each group
varies From one to 23 years), the linear trend estimates were multiplied by two and
divided by the number of years rated. The r-squares represent the amount of variance in the group's original ratings of members of Congress explained by the unidimensional coordinates for the members derived for the entire period.
Table 2 suggests a number of important conclusions. First, the variation in ratings from group to group which many critics find so damning apparently reflects the
variation in group positions on the liberal-conservative dimension. Second, with a
few exceptions, interest group positions over the 23-year period are remarkably
stable. These two findings suggest that the groups are fairly adept at selecting rollcall votes which accurately reflect the group's policy positions. Third, hypotheses 2
and 2A receive greater support than hypotheses 1and 1A. Both general-issue and
single-issue groups load highly on the derived dimension. Fourth, the recovered
positions of single issue-groups are generally less extreme than the dimensional positions of general-issue groups.
The recovered positions of the interest groups on the liberal-conservative
dimension in Table 2 suggest that the groups tend to be clustered at the ends of the
dimension; however, they are still spread over a considerable range. Most of the liberal scores fall in the range - ,300 to -1.200 while the conservative organizations
run from .400 to 1.500. Given the wide variance in positions on the dimension, the
variation in rating scores given by different groups becomes more understandable.
A moderate liberal group like Common Cause (-.460) will evaluate members of
Congress from a different perspective than Taxation with Representation ( -1.185).
Common Cause will tend to give scores of 100 to moderate liberals, scores of 50 to
liberals and moderates, and scores of o to 25 conservatives. Taxation with Representation, from its vantage point beyond the most liberal member of Congress, will give
scores of 80 to 100 to liberals and correspondingly lower scores as the members
~ ~ - ~to
o vthe
e conservative end of the scale. Thus, the differences in rating result, not
from any group's unique issue focus, but from their positions on the underlying
liberal-conservative dimension.
1
REHABILITATING THE RATERS 31
TABLE 2
Interest Group Positions, Linear Trends, and Degree of Fit
on a Liberal-Conservative Dimension
I
lange
le ac1110~tional
)e the
93rd
ld by
1985)
idual
'ludes
from
r the
lordir , the
l esti[roup
I and
varimidi1 ratis the
rith a
:ably
roll!ses 2
! and
rered
posi-
ative
~fthe
e libtions
I, the
able.
rs of
185),
50 to
Bengive
lbers
,not
lying
No, of
Group
TWR
SANE
ADA
CW
FCNL
CFNFMP
CFA
LCV
ACLU
NEWREP
UMW
NWPC
NCSC
AFT
COPE
BCTD
AFSCME
UAW
LWV
CWLA
BFW
NFU
LFR
NEA
CARTER
AFGE
CCAUSE
PFPJFK
JFK
PDPJFK
PDPLBJ
NFO
LBJ
PFPLBJ
PFPIKE
RIPON
IKE
NlXON
PDPIKE
ASCNSI
NFlB
CVVF
REAGAN
CCUS2
AFBF
Yearsa
locationb
Linear
Trendc
Yearly
Changed
R-Squarede
TABLE 2 (continued)
Group
CC
FORD
CCUS
CSFCD
CV
NASC
CSFCA
CSFCE
ACU
AC A
CSFCS
LL
LF S
NTU
No, of
Yearsa
23
3
7
2
2
4
3
2
9
23
2
11
1
10
~ocation~
1.077
1.082
1.111
1 .I 55
1.156
1 .I 71
1.188
1.196
1.258
1.262
1.264
1.388
1.421
1.471
Linear
Trendc
-0.030
0,010
-0.107
0.000
0.000
0.043
-0.034
0.000
0.034
-0.033
0.000
-0.006
0.000
-0.096
Yearly
Changed
-0.003
0.007
-0.031
0.000
0.000
0.022
-0.023
0.000
0.008
-0.003
0.000
-0.001
0.000
-0.019
R-Square@
0.897
0.588
0.770
0.847
0.831
0.847
0.909
0.876
0.890
0.896
0.855
0.809
0.855
0.335
-
Source: Dimensions of Interest Group Evaluation of Congress Study by Keith T. Poole and Howard
Rosenthal. Used with authors' permission.
aNumber of rating years in study.
bLocation of group on liberal-conservative dimension. Negative is liberal, and positive is conservative.
CDerivedlinear trend. Represents 112 of shift over the rating years for that group.
dYearlyshift in position. Calculated by multiplying linear trend by t w o and dividing by the number of
years.
@Ther-squared fit between actual ratings given each member of Congress by each group and the
predicted ratings derived from the unidimensional solution.
Moreover, group positions on the dimension are remarkably stable. The average yearly change that most groups make is relatively small. The absolute value of
the yearly change for all groups with more than two rating years in the data set is
.026. If only groups with more than ten rating years in the data set are included, the
average yearly change is ,019, A study by Daniels (1987) suggests that the median
number of years that the average group has rated is 12 years. If the average yearb
shift in Table 2 is representative of the larger population of rating groups, the average shift over a 12-year period would be .312, or 11.7 percent of a 2.656 unit dimension. This is not a substantial shift in position.
These first two conclusions have important implications for evaluations of the
accuracy of interest group ratings. If the groups are able to select congressional rollcall votes with enough precision to locate themselves consistently at the same point
on the liberal-conservative dimension over time, the ratings are probably fairly accurate representations of the issue positions of members of Congress. Some critiisms about the representativeness of the sample of votes that interest groups select
from the larger pool of all roIl-call votes can be made; however, Poole and Daniels
(1985: 394) have demonstrated for the 96th Congress that votes which are not
selected by any interest group are predicted just about as well by a single, liberalconservative dimension as roll-calls that are selected. In fact, non-selected roll-calls
1
REHABILITATING THE RATERS 33
%re@
97
88
70
47
31
47
09
76
90
96
55
09
55
35
)ward
i
con-
ber of
i d the
averue of
set is
1, the
?dian
early
avermen-
~f the
roll?oint
y accriti;elect
~niels
t not
,eralcalls
are predicted with nearly as much accuracy as roll-calls that are selected by five or
more groups. Even the selection of a small number of votes to calculate ratings
seems not to disturb this stability. For example, Bread for the World, a hunger group
which tends to find five or fewer votes in each house on which to base its ratings,
varies an average of ,033 units per year, only slightly higher than the average for all
groups. Thus, the second type of distortion does not seem to occur with much fiequency .
In general, the ratings of single-issue groups fit the derived dimension as well as
the ratings of general-issue groups. Since not all groups are issue-based, for this
the rating groups were classified into one of three types: economic, general
issue, or single-issue6 The group classifications are noted in Table 1.Mean fits were
calculated for each group. The mean r-squared for the 15 general-issue groups was
.708 with a standard deviation of .230. The comparable figures for the ten singleissue groups were .757 and ,069, and for the 13 economic groups were .716 and .137.
Single-issue
groups fit the dimension better than either general-issue or economic
groups. The fits for single-issue groups also had less variation from group to group.
The mean fit for general-issue groups was lowered by the poor r-squares for Common Cause, the Ripon Society, and the National Taxpayers Union.
Moreover, the dimensional positions of the various types of groups do not support the arguments of extremity made by rating critics. The mean ideological location of each of the three classifications of groups can be calculated by taking the
average of the absolute values of the group locations reported in Table 2. For
economic groups, the mean location is ,758. For general-issue groups the comparable figure is .990, while for single-issue groups it is .879. In other words, the most
moderate groups on the average are economic groups. Single-issue groups are somewhat more ideological in their positions, but the most extreme groups are the
general-issue groups. In fact, the issue positions of most general issue groups are
more extreme than the positions taken by the members of Congress.
DISCUSSION
In general, both groups and members of Congress appear to occupy a highly constrained issue space. Single-issue and general-issue groups define this space with
equal facility. If the universe of hypotheses 1and 1A were an accurate representation of roll-call votes and interest group ratings, those ratings would only imperfectly reflect the underlying dimension. In statistical terms, the r-square for those ratings
would be very low for some groups, On the contrary, Table 2 suggests that most
groups fit a single underlying dimension very well. More importantly, the hypothesized distortion by single-issue group ratings does not materialize. Single-issue
groups fit better than general-issue and economic groups and have smaller variations in fit. In addition, single-issue groups do not occupy as extreme a position on
the dimension as general issue groups. As a result, use of a single-issue group's
ratings to characterize the general issue space is not likely to lead to serious distortion (type 1).
However, this conclusion does not totally absolve interest groups of the charge
of distortion. Two sources of misrepresentation remain. The selection of fewer than
34 1 CONGRESS & THE PRESIDENCY
approximately five roll-call votes to calculate the ratings does introduce some error,
although the example of Bread for the World suggests that this source of inaccuracy
is not serious. More important is the potential for exaggeration produced by the fact
that most groups and members of Congress do operate within a common space.
long as a group matches its claims of narrow issue focus with vote selections which
match these issue interests, the resulting ratings are not likely to be distorted. m e n ,
however, a group uses a broad ideological perspective to select issues which purport
to describe a narrow and unique set of issue interests, the potential for serious misrepresentation is present. For example, if a group such as Christian Voice claims to
be rating on "family issues" and includes roll-call votes on the federal budget and
defense spending in its ratings, the resulting member evaluations may more closely
reflect a general ideological point of view than the claimed focus on family-oriented
policy.
The unidimensional issue space may seem surprising, but close examination
suggests that should not be unexpected. Although interest groups pursue a multitude
of goals, the individuals who join them tend to be drawn from the same small,
politically-motivated subset of the American population. Converse's (1964)classic
study of mass and elite issue constraint clearly suggested that the elite subset of the
population had more constrained political attitudes than the general population.
Thus, evidence of considerable constraint among interest group ratings should not
be surprising.
In addition, the groups themselves are operating within the constrained
universe of Congressional roll-call votes. Ratings are calculated on the basis of available roll-call votes which mirror the groups' positions. Hence, the groups are drawing upon a common universe. Research by Poole and Daniels (1985) and Poole and
Rosenthal(1986)has already demonstrated that the roll-calls in the period from 1959
to 1981 share a common underlying dimension. Evidence from the 95th Congress
(Poole and Daniels, 1985) suggests that this common dimension is shared not only
by roll-call votes selected by interest groups, but roll-call votes which are not
selected as well. Given this common underlying dimension, constraint in group
ratings based on congressional roll-calls is not unexpected.
CONCLUSION
The expansion of rating activity and the nature of the rating process have made the
vote evaluations much more controversial. Questions of roll-call selection, vote
direction, weighting, and the calculation of a single summary score have led to
charges of distortion, especially by members who receive low ratings in policy areas
which are of some importance to their constituents. These concerns have been
magnified by the fact that the rating groups use their ratings to provide information
to members, enhance the group's lobbying influence, and direct campaign contributions.
In general, these charges have centered on the unrepresentativeness of the rollcall votes on which the groups base their ratings. Particular criticism has been
directed at groups with a narrow issue focus whose selection procedures and narrow
policy interests supposedly make their ratings especially distorting, This article identified three kinds of distortion which ratings might produce. These types of distor-
REHABILITATINGTHE RATERS
'rror,
rracy
"act
e. AS
thich
hen,
rport
misns to
: and
osely
rnted
ation
itude
mall,
lassic
)f the
[tion.
d not
ined
.
ivail-
Irawand
1959
gress
only
2 not
Iroup
le the
vote
ed to
areas
been
ation
:ribu-
1 35
tion suggest two different kinds of relationships among issues, members, and
groups. The first is a multidimensional universe in which general-issue groups load
on general dimensions and narrowly-focused groups load on specific factors. The
second type of construct is a common space of low dimensionality in which all
groups are accurately located. The evidence discussed in this article generally supports the second model. For the most part, interest groups appear to be evaluating
members of Congress from their positions on a single liberalconservative dimension. Variations in ratings across groups reflect their differences in position on the
dimension rather than arbitrary selection of roll-call votes. By the same token, the
groups'positions on the dimension are remarkably stable over time. Both findings
suggest that the rating groups are very accomplished at their roll-call selection. Nor
does the evidence support the contention that single-issue groups are more arbitrary
in their selection criteria. These groups do not load on separate evaluative dimensions; rather, they appear to be as unidimensional as general-issue groups. Overall,
the interest group ratings appear to be a reasonably accurate representation of the
members' positions on an underlying liberal-conservative dimension. However, the
for distortion still exists if the group claims a narrow focus, but uses a
broad ideological perspective to select roll-calls.
Notes
The author wishes to thank Keith T. Poole and Howard Rosenthal for their permission to use the
data from their Dimensions of Interest Group Evaluation of Congress study. Any interpretations are
strictly the author's and should not be attributed to either Poole or Rosenthal.
1Telephone conversation with Katharine Vogel, Archivist, George Meany Memorial Archives, AFLCIO, Oct. 20, 1986.
2Thetotal number of rating groups is derived from Hays and Miller (1984)and Poole and Rosenthal
(1986).
jThis finding is reinforced by Poole (1981) and Weyrich (1980).
4Much of the following discussion is developed from Poole and Rosenthal (1986).
more complete discussion of the unfolding algorithm can be found in Poole and Rosenthal(1986:
30-36). See also Poole (1981, 1984).
6Corporations, trade associations, professional associations, unions, and farm groups were classified
ratings covered more than one general issue area or
as economic groups. All non-economic groups
social group were classified as general issue groups. All non-economic groups whose ratings covered only
one general issue area or social group were classified as single-issue groups. Congressional Quarterly and
New Republic were excluded from these
because they were journalistic enterprises rather
than lobbying organizations. Also
were the second 1980 Chamber of Commerce scale and the
defense, social, and economic scales for the Committee for the Survival of a Free Congress. Only the
general scale for the latter was used. These exclusions were done to ensure that each group was only represented once in the calculations.
References
I
roll-
been
rrow
idenistor-
Converse, Phillip. 1964. "The Nature of Belief Systems in Mass Publics," in David E. Apterr (ed.),
ideology and Discotttent. New York: Free Press.
Daniels, R. Steven. 1987. 'The Rating Game: Interest Groups and Congressional Roll-Call VotiW."Presented at the annual meeting of the Midwest Political Science Association, Chicago, IllinoisElimbeth. 1983. Po/ifics and Money: The New Road to Corruption. New York: Macmillan.
36 1 CONGRESS & THE PRESIDENCY
Fowler, Linda L. 1982. "How Interest Groups Select Issues for Rating Voting Records of Members of the
U.S. Congress." Legislative Studies Quarterly 7 (August): 401-413.
Hays, Janet, and Rod Miller. November 1984. Organizations that Rate Members of Congress on their
Voting Records. Washington: Congressional Research Service.
Hoadley, John. 1980. 'The Dimensions of Congressional Voting, 1971-1978: Some Preliminary consider.
ations," Presented at the annual meeting of the Midwest Political Science Association, chicago, 11.
linois.
Keller, Bill. 1981. "Congressional Rating Game is Hard to Win." Congressional Quarterly Weekly Report
39 (March 21): 507-512.
Kritzer, Herbert M. 1978. "Ideology and American Political Elites." Public Opinion Quarterly 42
(Winter): 484-502.
Libby, Orin G. 1897. A Plea for the Study of Votes in Congress. Annual Report of the American Historical Association, Vol. I. Washington: GPO.
National Farmers Union. 1982. 1981 Farm Bill Did Things T O Farmers, Not FOR Farmers. National
Farmers Union Washington Newsletter 29 (February 5): 1.
National Federation of Independent Business. 1982. How Congress Voted: 97th Congress. San Mateo,
CA: National Federation of Independent Business.
National Taxpayers Union. 1984. The National Taxpayers Union Rates Congress: Congressional Spending Study, 98th Congress, 2nd Session, 1984. Washington: National Taxpayers Union.
Poole, Keith T. 1981. "Dimensions of Interest Group Evaluation of the U.S. Senate, 1969-1978."Ame+
can Journal of Political Science 25 (February): 49-67.
Poole, Keith T . 1984. "Least Squares Metric, Unidimensional Unfolding." Pschyometrika 49 (September):
311-323.
Poole, Keith T,, and R. Steven Daniels. 1985. "Ideology, Party, and Voting in the U.S. Congress, 19591980." American Political Science Review 79 (June): 373-399.
Poole, Keith T., and Howard Rosenthal. 1985. "A Spatial Model for Legislative Roll Call Analysis."
American Iournal of Political Science 29 (May): 357-384.
Poole, Keith T., and Howard Rosenthal. 1986. 'The Dynamics of Interest Group Evaluations of Congress." Presented at the annual meeting of the American Political Science Association, Washington,
DC.
Poole, Keith T., and Howard Rosenthal. 1988. 'The Spatial Stability of Congressional Voting: 17891985." Presented at the annual meeting of the American Political Science Association, Washington,
DC.
Ripon Society. 1984. Voting Ratings: First Session-98th Congress, 1984. Washington: The Ripon Society.
Sabato, Larry J. 1984. PAC Power: Inside the World of Political Action Committees. New York: W.W.
Norton.
Schneider, Jerrold. 1979. Ideological Coalitions in Congress. Westport, CN: Greenwood Press.
Sorauf, Frank J. 1984-85. "Who's in Charge? Accountability in Political Action Committees." Political
Science Quarterly 99 (Winter): 591-614.
Tesh, Sylvia. 1984. "In Support of 'Single-Issue' Politics." Political Science Quarterly 99 (Spring):27-44.
Weyrich, Paul M. 1980. 'The New Right: PAC5 and Coalition Politics," in Michael J. Malbin, (ed.1,
parties, Interest Groups, and Campaign Finance Laws. Washington: American Enterprise Institute.