Making Inferences from Attitude-Behavior Correlations Duane F. Alwin Sociometry, Volume 36, Issue 2 (Jun., 1973), 253-278. Your use of the JSTOR database indicates your acceptance of JSTOR's Terms and Conditions of Use. A copy of JSTOR's Terms and Conditions of Use is available at http://www.jstor.org/about/terms.html, by contacting JSTOR at [email protected], or by calling JSTOR at (888)388-3574, (734)998-9101 or (FAX) (734)998-9113. No part of a JSTOR transmission may be copied, downloaded, stored, further transmitted, transferred, distributed, altered, or otherwise used, in any form or by any means, except: (1) one stored electronic and one paper copy of any article solely for your personal, non-commercial use, or (2) with prior written permission of JSTOR and the publisher of the article or other text. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. Sociometry is published by American Sociological Association. Please contact the publisher for further permissions regarding the use of this work. Publisher contact information may be obtained at http://www.jstor.org/journals/asa.html. Sociometry 01973 American Sociological Association JSTOR and the JSTOR logo are trademarks of JSTOR, and are Registered in the U.S. Patent and Trademark Office. For more information on JSTOR contact [email protected]. 02001 JSTOR http://www.jstor.org/ Thu Aug 16 11:13:10 2001 Sociometry 1973, Vol. 36, No. 2, 253-278 Making Inferences from Attitude-Behavior Correlations* DUANE F. ALWIN University of Wisconsin, Madison Social psychologists have m a d e a n u m b e r of inferences from the relatively. low predictability of verbal attitudes for overt behavior. T h r e e such inferences having t o d o w i t h t h e stability of attitudes, the measurement adequacy of verbal reports of attitudes, a n d t h e extent t o which attitudes determine or influence behavioral outcomes are prominent i n t h e attitude-behavior research literature. A n explicit causal model representing the key features of these theoretical inferences is presented t o illustrate the assumptions made by these inferences and the problems associated with making such inferences i n bivnriate situations. I n order t o understand the dynamics of the attitude-behavior relationship i n specific substantive situations, t h e need t o go beyond such data is emphasized, and some m e t h o d s for doing so are discussed. Some empirical examples using data from t h e existing literature are provided t o illustrate t h e suggested approaches. THERELATIONSHIP BETWEEN attitudes and behavior has long been of interest to social psychologists. This interest has taken two dominant forms: 1) an interest in the prediction of behavior from verbal attitudes, and 2) a theoretical concern with the nature of the observed relationships between verbal attitudes and actual behavioral responses. Although the two concerns are sometimes confused, they are quite distinct. Prediction is not concerned with hypothetical constructs such as "underlying attitudes" * T h e author was supported by a National Institute of General Medical Sciences Training Program in Methodology and Statistics (GMO-1526) and by post-doctoral support from Professors William H. Sewell ant1 Robert M. Hauser during the writing of this paper. Data analyses reported in the paper were supported primarily by a grant to Professor Sewell from the National Institutes of Health (M-6275). T h e author wishes to thank numerous persons for making helpful suggestions on earlier drafts of this paper, particularly Charles Susmilch, Robert M. Hauser, L. Edward Wells, and - the anonymous referees for Sociometry. T h e author, however, bears full responsibility for the contents of the paper. Computer facilities were provided by the Madison Academic Computing Center. - - 254 SOCIOMETRY -one wishes only to predict a person's behavior on the basis of knowledge of his stated attitudes. A theoretical concern, on the other hand, wishes to ,specify the causal relationships among the relevant variables, including unmeasured hypothetical constructs, so that an understanding or interpretation of the observed relationships can be reached.1 The present paper is concerned with the theoretical specification of the relationship between verbal attitudes and behavior, rather than with prediction. A major problem which is addressed involves the types of theoretical inference one can draw from an empirical relationship between measured (verbal) attitudes and overt behavior. T h e typical empirical situation which one encounters in the attitude-behavior research literature is a relatively low statistical relationship between a given verbal attitude and a behavioral response. A number of theoretical inferences are possible given this type of empirical situation, and I argue that given such a bivariate relationship there is virtually no way of deciding which of the several inferences is correct. My argument assumes a causal specification of the variables involved-verbal attitude, underlying attitude, and behavior. I argue further that new research strategies are required to assemble data which will begin to answer some of the theoretical issues involved in the attitude-behavior area. Before presenting the formal argument I will discuss the prediction and theoretical approaches to attitudes and behavior and the need for a theoretical framework which explicitly states the relationships among the relevant variables. PREDICTION VERSUS THEORETICAL APPROACHES T O ATTITUDES AND BEHAVIOR I t is perhaps fair to say that, at the level of prediction, the ability of verbal attitudes to predict behavior has been relatively disappointing to attitude-behavior researchers. In a recent review of the literature in the area, Wicker observes: . . . it is considerably more likely that attit,udes will be unrelated or only slightly related to overt behaviors than that attitudes will be closely related to actions. Product-moment correlation coefficients relating the two kinds of responses are rarely above .30, and often near zero. Only rarely can as much as 10% of the variance in overt behavioral measures be accounted for by attitudinal data. In studies in 1 The distinction drawn here between the use of attitude notions in prediction of behavior and their use in theories of behavior roughly corresponds to the distinction which DeFleur and Westie (1963) make between a "probability" conception of attitude (involved primarily where prediction is the goal) and a "latent proces" conception of attitude (involved primarily where interpxtation is the goal). ATTITUDE-BEHAVIOR CORRELATIONS 255 which data are dichotomized, substantial proportions of subjects show attitude behavior discrepancies (1969:65). T h e predictive approach typically fails to make a distinction between "attitudes" and "verbal attitudes," and it should be made clear that Wicker, in the above cited passage, is concerned with verbal attitudes rather than attitudes as hypothetical constructs. I n a somewhat earlier article, Tittle and Hill (1967) note that the predictive adequacy of verbal attitudes for behavior depends upon 1) the particular measurement instruments used, 2) the extent to which the criterion behavior involved is within the person's "common range of experience," and 3) the extent to which the particular criterion behavior is a "repetitive behavioral configuration." Under such circumstances, the best association Tittle and Hill are able to obtain is an average of .543 (gamma) using a 15-item attitude scale. As Wicker observes, however, this is atypical of most attempts to predict behavior from verbal attitudes. On a theoretical level, the circumstances have not been much more encouraging, at least in terms of understanding the nature of the relationship between verbally stated (or written) attitudes and actual behavior. After reviewing the literature in the area, Deutscher concludes: We still do not know much about the relationship between what people say and what they do--attitudes and behavior, sentiments and acts, verbalizations and interactions, words and deeds. . . . Under what conditions will people behave as they talk? Under what conditions is there no relationship? And under what conditions do they say one thing and behave exactly the opposite? In spite of the fact that all of these combinations have been empirically observed and reported, few efforts have been made to order such observations (1966:242-243). As noted above, the present paper is concerned with the theoretical specification of the relationship between verbal attitudes and behavior. Such a theoretical specification can perhaps best be accomplished by viewing the relationship in terms of a set of causal assumptions relating the relevant variables involved, both measured and unmeasured. I t is my contention that the theoretical failures in the area of attitudes and behavior stem from a lack of complete causal specification among the relevant variables, as well as a general confusion over what the relevant variables should be. Attitude-behavior researchers rarely take seriously the theoretical model which underlies their research operations. For example, the distinction between attitudes and verbal reports of attitudes is important to consider. Since a verbal report does not always adequately reflect the true underlying attitude, the two are not necessarily isomorphic and we must allow for this in our model of the relationships among verbal attitudes, 256 SOCIOMETRY attitudes and behavior. This paper argues, among other things, that our theoretical notions regarding measurement error must be brought explicitly into the models of the relationships among verbal attitudes and behavioral responses. THE NEED FOR A THEORETICAL FRAMEWORK While the issues of prediction and theoretical specification are distinct, they are not unrelated. Their major contact arises from the theoretical inferences which attitude-behavior researchers draw from the relative inability of verbal attitudes to predict behavior. Three such types of inference have been drawn: first, that underlying attitudes are not stable over time; second, that verbal attitudes are not entirely adequate as measures of underlying attitudes; and third, that attitudes are not powerful determinants of behavior. Two of these inferences are made by Wicker, who, states: . . . there is little evidence to support the postulated existence of stable underlying attitudes within the individual which influence both his verbal expressions and his actions . . . and the assumption that feelings (attitudes) are directly translated into actions has not been demonstrated (1969:75). One should keep in mind that Mricker's conclusion is based, for the most part, on evidence of bivariate relationships between verbal attitudes on the one hand and behavior on the other. The second inference noted above concerns the adequacy witli which verbal attitudes measure the "true" attitudes of individuals. The questions of both validity and reliability have been raised in this context, although the question of the validity of attitude measures (verbal attitudes) has most often been articulated. For example, LaPiere's often cited study was primarily an attempt to ,illustrate, through an example of attitude-behavior inconsistency, that questionnaire measurement of attitudes might not measure the attitude which is relevant to the behavior in question. He concluded his report witli the following statements: T h e questionnaire is cheap, easy, and mechanical. T h e study of human behavior is time consuming, intellectually fatiguing, and depends for its success upon the ability of the investigator. T h e former method gives quantitative results, the latter mainly qualitative. Quantitative measurements are quantitatively accurate; qualitative evaluations are always subject to the errors of human judgment. Yet it would seem far more worth while to make a shrewd guess regarding that which is essential than to accurately measure that which is likely to prove quite irrelevant (1934:237). Following LaPiere's essential conclusion, that attitude measures are more reliable than valid, Deutscher (1966, 1969) has written that the atten- ATTITUDE-BEHAVIOR CORRELATIONS 257 tion given to the measurement validity of verbal attitudes has been much too meager. Deutscher's (1969) argument stimulated a recent debate in the American Sociologist regarding the methodological quality of attitudebehavior research through the years (see Ehrlich, 1969; Ewens, 1969; Gordon, 1969; LaPiere, 1969; Ajzen et al., 1970; Lastrucci, 1970; and Tarter, 1970). I n spite of the LaPiere-Deutscher argument, it is also possible to infer poor reliability (consistency) of measurement, of both verbal attitudes and behavior, from the lack of substantial relationships among them. I t is well-known among social scientists that unreliability or random error of measurement attenuates hivariate relationships (see Bohrnstedt, 1970). Ehrlich (1969) makes this point in the debate which Deutscher recently stimulated. T h e third inference listed above, that attitudes are not powerful determinants of behavior, has been argued by Deutscher (1966) in two ways. First, he contends that notions of "cause and effect (and therefore of stimulus and response or of independent and dependent variable)" are basically untenable in science. Second, he asks whether other factors, like the definition of the situation and the influence of reference groups, are not more powerful determinants of behavior. This point of view is argued by Schwartz and Alwin (1971) on the basis of research evidence which points to the importance of situational factors in "unpatterned behavioral situations.", I n addition, a number of investigators, generally discouraged by the magnitudes of attitude-behavior correlations, have taken u p the task of studying the effects of other relevant variables on overt behavior (see Fendrich, 1967; Ajzen and Fishbein, 1969, 1970; Schwartz and Tessler, 1972; Warner and DeFleur, 1969; Wicker, 1971; McPhail, 1971; and Green, 1972). Each of the above three types of inference is consistent with the relatively low ability of verbal attitudes to predict behavior, and any or all of them would be plausible in any given case. Unfortunately, given a bivariate relationship between a verbal attitude and a behavioral response, there is no way we can discern which, if any, of the three inferences is correct. Given this impossibility, an inference from a near zero correlation that an attitude is not stable over time, or that the underlying attitude does not cause the behavioral response may be inco?,rect. Exactly how this might come about is demonstrated in the next section. Finally, some alternatives will be suggested which will offer a solution to the problem of identifying which, if any, of the above inferences is true in a given case. These suggested alternatives will then be illustrated. SOCIOMETRY SPECIFYING THE NATURE OF THE ATTITUDE-BEHAVIOR RELATIONSHIP Attitudes can be thought of as relationships between individuals and social objects which have both direction (positive-negative) and strength on a continuum of intensity. According to DeFleur and Westie (1963) most conceptions of attitude involve one of two views: a probability conception or a latent process conception. Both of these views involve a stimulus-response framework, a pattern of consistency in the attitude, and notions regarding the probability of recurrence of the attitude. The latent process conception, however, also posits an unobserved attitude which mediates the individual's responses, verbal and behavioral, to an attitudinal stimulus object. With this latent process conception in hand, we can formulate a model in which 1) there is a tendency for a person's underlying attitude to remain stable over time, 2) the person's attitude determines, with a certain amount of error, his verbal attitudinal response, and 3) the underlying attitude, among other factors, influences the person's behavior. Such a model can be represented in the diagram in Figure I , where the underlying attitude (A) is responsible for variation in both the verbal attitude (V) at time t and behavior (B) at time t 1. The model also posits an influence of the underlying attitude at time t (A,) on itself at I (A, + ,). Thus, altl~oughit is the same attitude, it is repretime t sented as two distinct variables because of the measurement lag between t and t 1. Finally, the model in Figure 1 allows for the influence of the verbal attitude (V) measured at time t on behavior (B) and the underlying attitude (A) at time t 1. The representation in Figure 1 follows the conventions of path analysis (Duncan, 1966; Land, 1968; Heise, 1968), which is quite useful in stating one's theoretical assumptions unambiguously. It is also consistent with previous treatments of unmeasured variables in path analysis (e.g., Heise, 1969; Hauser and Goldberger, 1971). The remainder of the paper will assume familiarity with linear structural equation models and path analysis. For the purposes of the argument presented below, it is important to indicate which parameters of the model in Figure 1 correspond to the key features of the three theoretical inferences outlined in the previous discussion. T h e parameter b represents the stability of the attitude (A) over time. If b equals unity, no individual's relative position in the attitude distribution has changed from time t to time t + 1 ; while if b is zero, the attitude distribution has undergone random fluctuations, with respect 1. The latter circumto the initial distribution, from time t to time t stance would indicate no intertemporal stability of the attitude (A), while + + + + + ATTITUDE-BEHAVIOR CORRELATIONS 259 FIGURE 1 Causal model for the relationship between a verbal attitude measure and a behavioral response. the former would indicate complete stability, allowing no influence from other sources. T h e parameter a represents the validity of the verbal attitude with respect to the underlying attitude. Validity, in this sense, is defined as the correlation between a true score and an observed score, in this case the attitude and the verbal attitude. This definition of validity is dependent upon a definition of the reliability of the verbal attitude as a 2 in the present model. In other words, reliability is defined as the squared correlation between the true and observed attitudes (see Lord and Novick, 1968:55-63).2 2 T h e definitions of reliability and validity of verbal attitudes presented here depend upon the assumption that there are two sources of variation in the verbal attitude: a reliable component representing the true underlying attitude, and an unreliable component representing random error of measurement. It is possible that the verbal attitude measure (V) reliably measures something in addition to, or instead of,the underlying attitude of interest. If the underlying construct is not A, then we must relabel it, SOCIOMETRY TABLE 1 Structural Equation Model for Figure I Structural Equations: +d l - az E t At+, = b At + d Vt + d l - [bz + dz + Zadb] W,,, Bt+i= c e Vt + d l - [cz + ez + Pcde] Ut+l Vt = a At &+I+ A third parameter, c, represents the extent to which the underlying attitude directly influences the behavioral response (B). If variation in the underlying attitude accounts for all of the variation in the behavioral response, c will equal unity. This is obviously unrealistic since the model in Figure 1 proposes other nonzero influences on B. In a given circumstance the underlying attitude may contribute very little to an observed behavior, given these other sources of variation, in which case c will be near zero. In order to conclude that an attitude does not determine a specific behavior this empirical situation must occur. In addition to the parameters a, b and c, the ,nodel in Figure 1 recognizes the potential effects of sensitization or reactivity of measurement on the underlying attitude (see Campbell and Stanley, 1963). This effect is represented by the parameter d. In a number of studies of attitude change which have gained recent popularity, this effect is of primary theoretical interest (see Bem, 1970 for a review of some of this literature). Finally, the effect of verbal attitudes on overt behavior is represented in Figure 1 by the parameter e. This is essentially the effect of "what people say" on "what they do." This parameter has been of theoretical interest to a number of consistency theories (see Kiesler et al., 1960:155-190). T h e model in Figure 1 is expressed in terms of a set of linear structural equations in Table 1. Given this model, we can write the correlation between the verbal attitude and the behavior as: Pve = c(d ba) e In the language of structural equation models, the model in Figure 1 and Table 1 is underidentified-there is only one equation in five unknowns- + + if possible. If more than one underlying construct is being measured by V (e.g., A and some sort of nonrandom response bias), then we must explicitly include both sources of variation in a theoretical model which includes a representation of the sources of variation in the verbal attitude (V). Unfortunately, such other sources d variation can rarely be specified, so we might overestimate a in a practical situation. This fact results in the principle that the square root of reliability is only an upper limit on validity (see Lord and Novick, 1968:72). ATTITUDE-BEHAVIOR CORRELATIONS 261 and we cannot generate a unique solution for the unknown parameters in terms of the observations.3 There is a simpler version of the model in Figure 1 perhaps more consistent with most theorizing regarding the relationship between attitudes and behavior. This simpler model assumes the parameters d and e to be zero in the population. Such assumptions can perhaps reasonably be made if the time lag between observations is sufficient to rule out the effects of those factors (such as sensitization and memory) wllich mediate the effects of stated attitudes at time t on both true attitude and actual 1. A potential problem with assuming d = e = 0 behavior at time t for longer time lags is that the same reasoning might lead one to assume that b also tends to zero for such time intervals. One might argue, however, that d and e are simply transient effects of measurement at time t, whereas b represents a more or less consistent tendency to respond. I n this simpler model, the correlation between the verbal attitude and behavior is: pvn = cba So expressed, it is clear that the assumption that d e = 0, does not assist us in identifying the other parameters of the full model-there are still three unknown parameters to estimate from just one empirical correlation. If we accept either one of the models discussed above, it should be clear that the magnitude of a particular verbal attitude-behavior correlation provides us n o evidence for the nzagnitude of any one of the param. eters of the model. Consider the simple model discussed above. T h e observation of a near zero correlation could indicate one of a number of things-any one, or more, of the parameters a, b, or c could be near zero. Such a correlation is not sufficient for the conclusion that one or some combination of a, b, or c is (are) equal to zero. I n other words, given either version of the model in Figure 1, it is inappropriate to argue (on the basis of a bivariate relationship of the type discussed above) that 1) attitudes are not stable over time (Wicker, 1969), 2) that verbal attitudes are necessarily poor measures of underlying attitudes (Lapiere, 1934; Deutscher, 1966, 1969), and/or 3) that attitudes do not influence behavior (Wicker, 1969; Deutscher, 1966; Schwartz and Alwin, 1971; and McPhail, 1971). Indeed, such inferences assume knowledge of the other parameters of the model-knowledge which is ordinarily unavailable in such two-wave + 3 For a brief introduction to the identification problem in structural equation models see Duncan (1972). 262 SOCIOMETRY studies. Taking the simpler model as an example, in order to conclude from a near zero correlation, r,,, tliat attitudes do not influence behavior (i.e., c = O), one implicitly makes the assumption tliat tlie verbal attitude is a perfect reflection of the underlying attitude (a = 1.0) and that tlie underlying attitude distribution is perfectly stable (b = 1.0). For example, suppose we have an independent measure of the reliability of an attitude scale (VJ, r,, = .5 (so that ci L .7). Suppose further that r,, = .35. Now, what range of values for b and c would generate such results given tlie estimated value of a? One possible result would be b = .5 and c = 1.0. Another would be b = 1.0 and c = .5. So, given such figures, both b and c must be in the range .5-1.0, within limits of sampling error. In this hypothetical case there is no real evidence for weak attitude-behavior effects or weak intertemporal stability of the attitude, yet the hypothetical value for r,, is not of particularly high magnitude. Still, this kind of evidence has led some observers to assert that attitudes do not influence behavior and that attitudes are not stable over time (cf. Wicker, 1969:75). IDENTIFYING THE PARAMETERS OF ATTITUDE-BEHAVIOR RELATIONSHIPS I n the preceding section a model was set forth for tlie bivariate relationship between a verbal attitude and a behavioral response. A correspondence was drawn between tlie parameters of that model and the three major theoretical inferences which are frequently rendered to interpret the typically low observed correlations between verbal attitudes and behavioral responses. It was argued that while the conclusions which come from these three types of inference may be true, there is no way of generating this knowledge from such bivariate relationships alone, since the model set forth is underidentified. What can be done to remedy tlie situation? In general, a solution involves the use of observations on additional variables-either verbal attitudes, behavioral responses or both. Unless additional data are collected, very' little can be done to enhance the identification of the parameters of the model in Figure 1 without drastically misrepresenting reality. Some alternate solutions are presented in this section. Suppose we are able to observe the verbal attitude at times t and t 1 and the behavioral response at time t 2. T h e model which represents this possibility is shown in Figure 2, and is stated in structural equation form in Table 2. This model assumes that dl = d, = e = f = 0; that the stability or instability of the attitude is constant over time intervals of equal size (b, = b,); and that the verbal attitude is an equally valid + + 263 ATTITUDE-BEHAVIOR CORRELATIONS FIGURE 2 Causal model for the relationships anzong two verbal attitude measures and a behavioral response. measure of the underlying attitude at both points in time (a, = a,).4 In addition, we must assume, as specified in Figure 2, that the residual terms E,, E, + W, + W, + , and U, + , are uncorrelated with each other and with other prior variables in the system. With the additional observation of the verbal attitude and the required assumptions noted in the preceding paragraph, it is now possible to arrive at unique estimates for a, b and c. Tlie estimation equations for this model are presented in Table 2. Solving first for b, we find, ,, ,, A b = r,:, / r2, = ab2c / abc Substituting this solution for b into the equation for r,, we have, r12 = a2 r13/ r:t3, 4 A new theoretical parameter, f, has been included in the model in Figure 2 to represent the effect of verbal attitude at time t on verbal attitude at time t 1. + SOCIOMETRY TABLE 2 Structural Equation Model for Figure 2 Structural Equations: Vt = a At d l z + ~ + + + - , At+>= b At wt+, Vttl = a Att1 d l -a2 Et+, Attz = b At+>+ - b2 Wtta Bt+z= c Attz d l c2 Ut+, Path Estimation Equations: (where Vt = XI; V ttl = X,; Btt2= X,) rIa= a2 b r,, = abzc r,= abc and solving for a, we find, A a= v r12 r23 / r13 Finally, using the equation for r2, we can solve for c: Suppose we observe the following relationships in the above situation (Figure 2): r12 = .3, r13= .2, and r23 = .3. These are of the order of magnitude which is typically observed in attitude-behavior research (see A A A Wicker, 1969:65). In this case a = .67, b = .67, and c = .67. Although the observed correlations in this hypothetical case may be considered relatively low (they account for less than ten percent of the variance), one would certainly not conclude, given the theoretical model, that the underlying attitude is unstable, that the attitude is not a viable determinant of the behavior, or that the verbal attitude is an inadequate measure of the underlying attitude.6 There are several other solutions to the identification problem which extend the research design in attitude-behavior research to include measurement at three points in time. For example, we can construct a 5 While the standard errors of these estimates are not developed here, they are expected to be large for small samples. Therefore, one would need large samples to obtain stable estimates. ATTITUDE-BEHAVIOR CORRELATIONS 265 model in which we have data on a behavioral response at times t and t 1 and a verbal attitude at time t 2. In such a case it would be necessary to assume the equality of the attitude-behavior effects (the c's) at times t and t 1, as well as the equality of the stability parameters over time (the b's). Estimates for such a model will be derived in the "empirical example" in the following section. If one has two measures of a behavioral response and one measure of a verbal attitude, or two measures of a verbal attitude and one measure of a behavioral response over time, a variety of linear models can be estimated in a straightforward fashion provided, the necessary assumptions are appropriate for the situation. The particular model developed here, in Figure 2, is perhaps the most useful for a number of reasons. First, from a purely methodological point of view, it is probably easier in most research to obtain two measures of a verbal attitude separated in time rather than two behavioral measures. Unless the researcher has access to a recurrent behavioral situation which is uniformly presented to the respondent at each point in time (e.g., in an experimentally contrived situation), the collection of behavioral data over time is problematic. Second, from the standpoint of making the assumptions necessary to identify the parameters of the models, the assumption that verbal reports of attitudes have no enduring effects on either the underlying attitude or later behavior (d = f = 0 in Figure 2) may be more palatable than the alternative assumptions that behavioral responses do not have any lasting effects on the underlying attitude and/or later behavior. In any case, the reader should be cautioned against applying any of the estimation procedures discussed here in situations where the assumptions involved are clearly not in accord with reality. + + + AN EMPIRICAL EXAMPLE I t is difficult to find empirical examples from the available literature on attitudes and behavior which conform to the longitudinal character of the present approach. Data are not routinely collected at three points in time. There is, however, at least one case in the existing literature in which the class of models presented above can be construed to fit a particular set of data-the Tittle and Hill study (1967). Although the data from this study were not explicitly collected under the guidance of a three-wave design, the variables involved in the study occur at multiple (> 2) points in time, and it is possible to use these data to illustrate the approach suggested above.6 6 The author wishes to thank Professors Charles Tittle of Florida Atlantic University SOCIOMETRY TABLE 3 Correlation Matrix for Selected Variables from Tittle and Hill (1967) Study Variable * Variable 1 2 3 4 5 6 7 1 2 1.oooo ,2440 .2020 ,2160 .OW0 ,1140 .0830 ...... 1 .OOOO .2150 .3790 .2080 .2420 ,1650 4 5 6 7 ...... ...... ...... ...... ...... ...... ...... ...... ...... ...... ...... ...... ...... 1 .OOOO ,5910 ,3510 ...... ...... ...... ...... ...... ...... 1 .OOOO .3570 1 .OOOO 3 1 .OOOO ,5730 ,3910 ,4380 ,2520 1 .WOO .6390 .6970 ,4350 ...... * T h e variable numbers are as follows: (1) vote prior to last election; (2) vote in last election; (3) self rating of attitude; (4) Likert scale of attitude; (5) Thurstone scale of attitude; (6) Semantic Differential of at,titude; and (7) Guttman scale of attitude. Tittle and Hill studied the relationship between verbal attitudes toward political participation and political activity. They obtained multiple measures of both dimensions and report the relationships (gammas) among the several measures. T h e verbal attitude measures were obtained from a questionnaire administered to 300 college students at a major university. Five attitude scales were constructed from the item responses: 1) a 15-item Likert type scale, 2) a ten-item Guttman scale, 3) a Thurstone scale with fifteen scale points, 4) a nine-item Semantic Differential scale, and 5) a single item self-rating. One measure of political behavior was obtained from student voting records in an election held one week prior to the administration of tlie questionnaire. Another measure of political activity was obtained by eliciting the respondent's report of his voting behavior for the previous four elections. Other informati011 regarding the student's political activity was obtained, but tlre present discussion will be confined to these seven measures. The student's "vote over time" is modified slightly for the present purposes. It is scaled to represent whether the respondent reported having ever voted prior to the last election.7 The correlation matrix for these seven variables is presented in Table 3. It is now possible to outline a three-wave design which may be taken to underlie these Tittle and Hill measures. In particular, tlie "vote over and Richard J. Hill of the University of Oregon for kindly making their data available for the purposes of estimating the present modets. 7 This was done for two reasons: 1) to alter the fact that the "vote over time" m a sure contains variation due to the respondent's vote in the fast election, and 2) to put the two variables in the same metric. ATTITUDE-BEHAVIOR C O R R E L A T I O N S 267 time" as modified here can be considered a measure of voting at time t (B,), the measure of the "record vote" in the last election can be considered a measure of voting at time t 1 (B,), and any one of the attitude measures can be considered as occurring at time t 2 (V). This model for the two behavioral responses and a single attitude is presented in Figure 3. This particular example affords the opportunity to illustrate not only the approach to estimating the theoretical parameters involved, but also for a situation which is different (but n o less viable) from that discussed i n Figure 2 in terms of the placement of the verbal attitude and behavior measures. I n order to identify the theoretical parameters of this model, it is necessary to assume no causation from B , to B, and from Bl and B2 to V, as well as n o inf uence of the behavioral responses on the underlying b2 = b and c1 = c, = c, attitudes. Further, we must assume that b , and that the disturbances on B,, B, and V are uncorrelated with each other and with prior variables in the model. + + w2 FIGURE 3 Causal model for the relationsltips among voting behavior at two points in time and a measure of verbal attitude toward political participation. SOCIOMETRY TABLE 4 Structural Equation Model for Figure 3 Structural Equations: Bt = c A, + d l > u t + At+, = b At - 62 Wt+l Bt+l=~At+l+dG~Ut+l At,, = b At+, -wt+, Vt+, = a At+, d l - az Et+, + + Path Estimation Equations: (where B, r,, = cz b r,, = c bz a r,, = c b a = X,; Bt+, = X,: V,,, = X$ T h e foregoing assumptions permit the derivation of the path estimation equations from the structural equations presented in Table 4. Solving the estimation equations for a, b and c in straightforward algebraic fashion we find: These formulae can be applied to the appropriate correlations among the variables in Table 3, where X1= B,, X, = B, and X, = Vj (for j = 1, 5). Thus, we can estimate five separate models-one for each separate attitude measure. T h e results of tllese estimation procedures appear in Table 5. The results of this approach provide a range of estimates for the parameters involved. The estimates clearly depend on the particular attitude TABLE 5 Pamirzeter Estimates for a Three-Wave Model of Attitudes and Behavior Based on the Tittle and Hill (1967) Study Attitude Measure Parameter Likert Guttman Self Sem. Diff. Thurstonc ATTITUDE-BEHAVIOR CORRELATIONS 269 measure used. T h e estimates range from .45 to 1.02 for a ; those for b from .43 to .94; and those for c from .51 to .75. With the possible exception of the estimates produced using the Likert attitude scale, the sample results are within reasonable bound^.^ Whichever set of estimates one prefers to take as reasonable, it is clear that none of the estimates is equal to zero, or is even near zero, as some of the theoretical speculation based on attitude-behavior correlations would perhaps lead us to expect. I t is important to bear in mind, however, that the interpretive weight one places on these estimates depends on the plausibility of the theoretical model. T h e fact that the estimates are within reasonable bounds does not confirm the model in any "theory testing" sense. We have simply estimated a model given several plausible assumptions and have found the estimates to be consistent with our expectations. T h e fact that we are not led to reject the model does not necessarily confirm it. For example, if the effect of behavior at time t has a nonzero effect on attitude at time t 1, our estimates of the just-identified model could still make sense even though the estimated stability effect (b) would be spuriously high. Likewise, if in the true model the behavior at time t has an influence on the behavior at time t 1 and/or the behavior at time t 1 has an influence on the verbal attitude at time t 2, the estimated effects, b and c, could both be biased. We should therefore keep these estimated effects in the proper perspective, given what we believe to be the correct model. + + + IDENTIFYING T H E PARAMETERS-AN + EXTENSION T h e approach suggested above is somewhat difficult to work with, especially within the context of the Tittle and Hill data set. One of the problems encountered is the necessity of assuming that c, = c, and b, = b, (see Figure 3 and Table 4) in order to make the model just-identified so that the theoretical parameters of interest can be estimated. These assumptions become problematic in the present context because the behavioral measures at times t and t 1 (B, and B,) are not equivalent. Specifically, B, represents the respondent's vote over time prior to the last election based on his self report, while B, represents the respondent's vote in the last election based on voting records. Moreover, the measurement intervals are not equidistant-times t 1 and t 2 are separated by one week, while a finite but indeterminable amount of time separates times t and t 1. While this failure to meet the strict assumptions set forth initially + + + + 8 Even so, there should be no reason to expect the sample estimates to be within the & 1.0 range given some sampling error. 270 SOCIOMETRY may not invalidate our previous findings, it would be preferable to work with a set of assumptions which does not require the equalities, at least insofar as the Tittle-Hill data are concerned. A second problem is that we do not have an overall estimate of the parameters involved (in particular b and c). Rather, the above approach yields five different estimates which depend upon the particular attitude scale used in the estimation procedures. It would therefore be convenient to have estimates for b and c which combine in some fashion the separate estimates presented in Table 5. A third problem which characterizes the previously outlined approach is that it does not make use of all the available information, vir. the interrelationships among the five attitude measures. The model presented in Figure 4 overcomes these three difficulties. In this model the effect of the attitude on behavior at time t (c,) is not necessarily constrained to be equal to the effect of the attitude on behavior at time t f l (c,). Nor is the stability of the attitude from times t to t 1 (b,) necessarily equal to the stability of the attitude from times t + 1 to -1 (i2i3j4j + b;\l/ t 1 bl A1 * A3 bA2 T W2 ' W3 FIGURE 4 Causal model for the relationships m a n g votittg behavior at t w o points in time and multiple measures of verbal attitude toward political participation. 27 1 ATTITUDE-BEHAVIOR CORRELATIONS + + t 2 (bz). Also, five measures of verbal attitudes at time t 2 are included, allowing for difference~in their validity coefficients (the a's) to be influenced by their intercorrelation. The model presented in Figure 4 is equivalent to a classical factor analysis model in which each measured variable has a nonzero loading on only one of three underlying factors, and zero loadings on all others. T h e model can be stated in matrix notation by: Tile variance-covariance matrix for the underlying factors, in standard score form, is as follows: Note that the model specifies three oblique (correlated) factors to account for the correlations among the observed variables, with the added restriction that the correlation between two of the factors equals the product of their correlations with the third. In particular, the model specifies that ~ A =~ A . ~pAZA3. This specification amounts to an hypothesis of a zero-order causal chain at the level of the underlying attitude. Finally, since the Likert (V2), Thurstone (V3) and Guttman (V,) scales involve 272 SOCIOMETRY overlapping content (Tittle and Hill, 1967:205-206), their errors (E,, E3 and E,) are allowed to intercorrelate freely in the model depicted in Figure 4. T h e reader will recall that the model estimated previously-see Table 4--was just-identified. In contrast to this, the present model is partially overidentified and partially underidentified. In particular, the model is overidentified with respect to the parameters a,, a,, a3, a4, a5, b2, and c,, but underidentified with respect to the parameters c, and b,. It is, however, overidentified with respect to the product of the parameters c, and b,. In order to identify these two parameters it will be necessary to assume that either b, = b2 or c, = c, (or both). It is possible to estimate the model under either set of assumptions and compare the results. In addition, one could simply estimate the product c,b,, ignoring the separate identification of the parameters. This would not entail a large sacrifice with respect to the original theoretical issues since we remain in a relatively strong inferential position vis-d-vis the identification of the parameters c2 and b,. It is possible to estimate the model under any of these conditions using the Joreskog confirmatory factor analysis (CFA) approach (Joreskog, 1970; Joreskog et al., 1970). The CFA procedure allows the estimation of factor models in which some of the parameters are constrained to be equal to zero. It also allows some of the parameters to take on identical but unknown values, estimating the unknown parameters under these and the null constraints.9 In the present case a number of the parameters of the B mqrix are set equal to zero, and some constraints are placed on the variakce-covariance matrix for the factors, E (FF') (see above).lO The Joreskog program provides maximum-likelihood estimates for the identifiable parameters. 9 T h e author wishes to acknowledge the Center for Demography and Ecology, Department o f Sociology, University o f Wisconsin for providing support for the Joreskog, Gruvaeus and van Thillo computer program and the efforts o f Robert M . Hauser, William M . Mason and Richard T . Campbell for obtaining the program and making it operational on the Univac 1108 at the MACC. lo In the present situation we will settle for an estimate o f the product c,b,. T h e n given some assumptions about one or the other o f the two parameters, we will be able to deduce the value of the other. T h i s is not an approach to estimation in the general underidentified situation, i.e., we cannot estimate both parameters under some assumption regarding the value o f the other. This would involve the obvious contradiction that a parameter can take on one value for one purpose and a potentially different value for another. Only i f one is in a position to follow a strong set o f assumptions with regard to the value o f one o f the parameters, e.g., b, = b,, is identification really possible in the present case. 273 ATTITUDE-BEHAVIOR CORRELATIONS T h e following estimates are produced by the application of the CFA procedures to the Tittle-Hill data for the above model (Figure 4): A B= ?(.657) .OOO .OOO .000 .OOO .OOO .OOO .OOO .662 .OOO .000 .OOO .OOO .OOO .OOO> .OOO .581 .968 , E (F^Ff)= .744 .732 .468, 1.ooo (.556) 1.000 (.311) .560 1.000 I I .OOO ,000 ,000 .OOO .OOO . O O O ~ .749 .ooo .ooo .ooo .ooo .ooo ,000 .OOO .814 .OOO .OOO .OOO .OOO and U = .000 ,000 .000 .249 ,000 .000 ,000 . .OOO .OOO .OOO .OOO .668 .OOO ,000 .OOO ,000 .OOO ,000 .OOO .681 .OOO L . O O O .OOO .OOO .OOO .OOO .OOO .884, -.930 .ooo A T h e estimate of the product clbl is .368, so assuming b1 = b, = .56 an estimated value for c1 is calculated to be .657 (set off by parentheses in the A A A matrix B). Similarly, if we assume cl = c, = .662, b1 is equal to .556 [set off by parentheses in the matrix E (fit)]. T h e value for b;b, is calculated, A in either case to be .311 [also set off by parentheses in the matrix E (FFf)].ll These estimates generate the correlation matrix for the variables presented in Table 6.12 T h e original correlation matrix (Table 3) minus this reproduced correlation matrix yields the residual matrix presented in Table 7. T h e estimates of the parameters in B suggest a number of inferences. First, the underlying attitude has a substantial effect on the behavioral A A A Since the values of c, and b, are nearly identical whether we obtain them by "assumption" or by "deduction," their underidentification does not present problems of interpretation. \Ye should keep in mind, however, that if we assume positive values for both parameters, they may take on a large number of values which would satisfy the estimate of their product. Indeed, any two positive values between .368 and 1.0, the product of which equals .368 will satisfy the requirements of the model. I n any case, we are in a relatively strong inferential position with respect to the parameters b, and c, (see text above), so the argument being made does not depend on exact estimates of b, and c,. 12 T h e following estimates are obtained for the correlated error terms: I-,,,,=-.489, rE2E5=-.082, and rc3E5=.005. These correlations enter into the calculation of the reproduced correlation matrix. 11 274 SOCIOMETRY TABLE 6 Reproduced Correlation Matrix for Selected Variables from Tittle and Hill (1967) Study Variable * Variable 1 2 3 4 5 6 7 * 1.000 .244 .120 .200 .154 .I51 .097 ..... ..... 1.000 ,216 .359 .276 .272 .174 1.000 .563 .432 .426 .272 ..... ..... ..... ..... 1 .OOO .639 .709 .435 ..... ..... ..... ..... 1 .OOO ,545 ,351 ..... ..... ..... ..... ..... ..... ..... 1 .OOO ,342 1 .Om ..... ..... ..... ..... See Table 3 for variable names. response at time two (.662), and also at time one (.657) if we assume b, = A b,. These effects appear 'to be equal in magnitude. Second, the attitude measures have validity coefficients of generally sizeable magnituderanging in value from .47 to .97-which have essentially the same relaA tive magnitudes as the coefficients for a in Table 5. Third, the estimate of the stability parameter between times two and three (.56) suggests that the underlying attitude maintains some stability (consistency) over time. A Further, if we assume that c, = c,, the estimates for the b's suggest that the rate of change in the attitude is approximately constant over intervals which encompass about the same amount of time. While none of these inferences is assumption-free, the ones regarding the equality of the c's and the equality of the b's are interdependent. We cannot logically infer one set of equalities from the sample data without assuming the other TABLE 7 Residual Correlation Matrix for Selected Variables from Tittle and Hill (1967) Study Variable Variable 1 1 2 3 4 5 6 7 2 - .000 -.om .... -.mo .082 ,016 - ,064 - ,037 -.014 .020 -.068 - .030 -.009 - .001 3 4 5 6 7 .... .... .... .... .... .... .... .... .... .... .... .OOO .010 -.04] .012 -.020 * See Table 3 for variable names. * .... .... -.OOO ,000 -.012 -.OOO .... -.OOO .046 .000 .... .... .000 .015 .... .... .... .... -.OOO ATTITUDE-BEHAVIOR CORRELATIONS 275 set of equalities to be true in the population model. In fact, we can only really have it one way from the standpoint of making inferences from sample data. Whatever the case, the estimates of the parameters of the theoretical model imposed on the data reproduce the correlations among the variables reasonably well (see Table 7). In no case are the reproduced correlations off more than .08 from the observed correlations, and most differ less than + .03. This represents a reasonably good fit to the data.13 , SUMMARY AND CONCLUSION T h e above presentation has attempted to speak to the theoretical issues involved in attitude-behavior research. We have reviewed the theoretical inferences which have been made from the general inability of verbal attitudes to predict behavior, and it has been argued that the major conclusions which have been drawn from this predictive state of affairs are inappropriate given the data on which the predictive information rests. Hopefully, the present analysis will caution persons against making theoretical inferences on the basis of data which are not adequate for these purposes. It is also important to emphasize that general statements regarding "the extent to which attitudes determine behavior" for example, should perhaps not be made outside the context of a specific substantive domain. T h e argument presented here is based on a theoretical model (see Figure 1) which is taken to be "implicit" in much of the discussion of the relationship between verbal attitudes and behavior responses. It is crucial to understand that even if one rejects this model as the "implicit" model in any given case, alternative bivariate models which include the issue of measurement error for verbal attitude measures and which allow for instability in the attitude over time, will also be underidentified. Unless one is prepared to assume knowledge of certain of the key parameters, e.g., perfect measurement reliability and/or stability of the attitude distribution over time, the argument set forth above stands, even though the model might oversimplify reality in the sense that it leaves out other sources of variation in both verbal attitudes and behavior. Given the theoretical assumptions which are taken to be "implicit" in much attitude-behavior research, it becomes clear that the type of data 13 A te5.t for the goodness of fit discussed in Joreskog (1970:241) utilizes a test statistic which is distributed as X 2 for large samples, assuming multivariate normality for maximum-likelihood estimates. T h e result of this test in the present case is ~ z = 15.6325 with 10 degrees of freedom. T h e reader is invited to decide for himself regarding the fit of the model in these terms. 276 SOCIOMETRY typically collected to analyze the theoretical relationships is inadequate to answer the critical theoretical issues. Therefore, some alternative approaches are suggested which are explicit with regard to theoretical assumptions. While the suggested approaches make the theoretical issues clear, they should be viewed as first approximations to an adequate theoretical representation of the relationship between attitudes and behavior. The present models do not include specific sources of variation, other than the underlying attitude, in both verbal attitudes and behavioral responses. If we believe the measurement error to be random, this lack of specification with respect to verbal attitudes is of no consequence. However, other sources of variation in the determination of behavior may be of interest since they are taken to be of major importance (cf. Wicker, 1971). Therefore, the next step in building a model which represents the theoretical relationship between attitude constructs and behavior is to specify first the variables which, along with underlying attitudes, cause variation in behavior-and then to include these variables in models, similar to those above, which make clear the theoretical assumptions about the ways in which the variables are'related. These tasks must by uecessity be carried out in a separate paper. Using data from the Tittle and Hill (1967) study on attitudes toward political participation and political activity, two theoretical models are estimated: 1) a just-identified three-wave model with behavior (voting) measured at two points in time and a verbal attitude measured at a third point in time, and 2) a partially overidentified model similar to the first which includes five measures of verbal attitudes at the same point in time. These analyses are carried out primarily to illustrate the suggested alternative approaches. For a number of reasons, the partially overidentified three-wave model is superior to the just-identified one, largely because fewer assumptions are required with respect to the theoretical parameters of the model. Even so, within the context of the Tittle and Hill data, both sets of estimates are reasonable given the model, and the estimates for the overidentified model suggest that the equality assumptions for the just-identified model might not be unrealistic. Finally, the estimates for these models of political attitudes and behavior do not support certain genkral conclusions which are made with regard to the stability of attitudes over time, the validity and reliability of verbal reports of attitudes, and the extent to which attitudes determine behavior. REFERENCES Ajzen, I. and M. Fishbein 1969 "The prediction of behavioral intentions in a choice situation." Journal of Experimental Social Psychology 5 (October):400-416. ATTITUDE-BEHAVIOR CORRELATIONS 277 1970 "The prediction of behavior from attitudinal and normative variables." Journal of Experimental Social Psychology 6 (October):466-487. Ajzen, I., R. Darroch, M. Fishbein and J. Hornik 1970 "Looking backward revisited: A reply to Deutscher." American Sociologist 5 (August):267-273. Bem, D. 1970 Beliefs, Attitudes and Human Affairs. Belmont, Calif.: Brooks-Cole (Wadsworth). Bohrnstedt, G . W. 1970 "Reliability and validity assessment in attitude measurement." Pp. 80-99 in G. Summers (ed.), Attitude Measurement. Chicago: Rand McNally. Campbell, D. T. and J. Stanley 1963 "Experimental and quasi-experimental designs for research on teaching." Pp. 171-247 in N. Gage (ed.), Handbook for Research on Teaching. Chicago: Rand McNally. DeFleur, M. L. and F. R. Westie 1963 "Attitude as a scientific concept." Social Forces 42 (October):17-31. Deutscher, I. 1966 "Words and deeds: Social science and social policy." Social Problems 13 (Winter):235-254. 1969 "Looking backward: Case studies on the progress of methodology in sociological research." American Sociologist 4 (February):35-41. Duncan, 0. D. 1966 "Path analysis: Sociological examples." American Journal of Sociology 72 (July): 1-16. 1972 "Unmeasured variables in linear models for panel analysis." Pp. 36-82 in H. L. Costner (ed.), Sociological Methodology, 1972. San Francisco: Jossey-Bass. Ehrlich, H . J. 1969 "Attitudes, behavior, and the intervening variables." American Sociologist 4 (February):29-34, Ewens, W. L. 1969 "Looking backward through a glass darkly." American Sociologist 4 (August): 251. Fendrich, J. M. 1967 "Perceived reference group support: Racial attitudes and overt behavior." American Sociological Review 32 (December):960-970. Fishbein, M. 1967 "Attitude and the prediction of behavior." Pp. 477-491 in M. Fishbein (ed.), Attitude Theory and Measurement. New York: John Wiley and Sons. Gordon, L. 1969 "On attitude-behavior correlations." American Sociologist 4 (August):250-251. Green, J. A. 1972 "Attitudinal and situational determinants of intended behavior toward blacks.!' Journal of Personality and Social Psychology 22 (April):13-17. Hauser, R. M. and A. S. Goldberger 1971 "The treatment of unobserved variables in path analysis." Pp. 81-117 in H. L. Costner (ed.), Sociological Methodology, 1971. San Francisco:Jossey-Bass. Heise, D. R. 1969a "Problems in path analysis and causal inferences." Pp. 38-73 in E. F. Borgatta (ed.), Sociological Methodology, 1969. San Francisco:Jossey-Bass. 278 SOCIOMETRY 1969b "Separating reliability and stability in test-retest correlation." American Sociological Review 34 (February):93-101. Joreskog, K. G. 1970 "A general method for analysis of covariance structures." Biometrica 57 (#2): 239-25 1. Joreskog, K. G., G. T. Gruvaeus and M. van Thillo 1970 "ACOVS--a general computer program for analysis of covariance structures." Educational Testing Service, Princeton, New Jersey. RB-70-15 (February). Kiesler, C. A., B. E. Collins and N. Miller 1969 Attitude Change: A Critical Analysis of Theoretical Approaches. New York: Wiley. Land, K. 1969 "Principles of path analysis." Pp. 3-37 in E. F. Borgatta (ed.), Sociological Methodology, 19G9. San Francisco: Jossey-Bass. LaPiere, R . 1934 "Attitudes versus actions." Social Forces 13 (December):230-237. 1969 "Comment on Irwin Deutscher's Looking Backward." American Sociologist 4 (February):41-42. Lastrucci, C. L. 1970 "Looking forward: T h e case for hard-nosed methodology." American Sociologist 5 (August):273-275. Lord, F. M. and M. R. Novick 1968 Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley. McPhail, C. 1971 "Civil disorder participation: A critical examination of recent research." American Sociological Review 36 (December):1058-1073. Schwartz, M. and D. F. Alwin 1971 "The evaluation of social action." Pp. 609-634 in E. 0.Smigel (ed.), Handbook on the Study of Social Problems. Chicap: Rand McNally. Schwartz, S. H. and R. C. Tessler 1972 "A test of a model for reducing measured attitude-behavior discrepancies." Journal of Personality and Social Psychology 24 (November):225-236. Tarter, D. E. 1969 "Toward prediction of attitude-action discrepancy." Social Forces 47 (June): 398-405. 1970 "Attitude: T h e mental myth." American Sociologist 5 (August):276-278. Tittle, C. R. and R. J. Hill 1967 "Attitude measurement and prediction of behavior: An evaluation of condi. tions and measurement techniques." Sociometry 30 (June):199-213. Warner, L. G. and M. L. DeFleur 1969 "Attitude as an interactional concept: Social constraint and social distance as intervening variables between attitudes and action." American Sociological Review 34 (April):153-169. Wicker, A. W. 1969 "Attitudes versus actions: T h e relationship of verbal and overt behavioral responses to attitude objects." Journal of Social Issues 25 (Autumn):41-78. 1971 "An examination of the 'other variables' explanation of attitude-behavior inconsistency." Journal of Personality and Social Psychology 19 (July):18-30.
© Copyright 2026 Paperzz