Use of the Personal Report of Communication Anxiety (PRCA-24) in Japanese Contexts: Reliability, Structural Integrity, and Response Validity Paper presented at the National Communication Association Convention San Francisco, California, USA November, 2010 Steven L. Renshaw Kanda University of International Studies Chiba, Japan Abstract The Personal Report of Communication Anxiety (PRCA-24) has been used extensively in a variety of cultural contexts to measure apprehension in various communication situations. This study examined a Japanese translation of this scale in order to assess reliability and aspects of validity including intrascale correlations, structural integrity, and measurement continuity. The scale was administered to 350 Japanese college students. Alpha coefficients and item-scale correlations for the instrument and subscales matched parameters found in development of the English version. While results of these classic reliability measures were comparable, confirmatory factor analysis showed weak goodness of fit relative to subscales, and Rasch analysis indicated that overall use of the scale might fail to fully cover or discriminate levels of the trait. Using classical assessments, the scale appears to provide a reliable and consistent measure of communication anxiety. However, its apparent weak structural integrity and inability to fully cover the purported trait indicate that caution should be exercised in its use for research in Japan. The uncritical summing of items or summed groupings of the subscales, may fail to differentiate or integrate factors that are particularly relevant to the underlying trait of communication anxiety as it is found in Japan and other cultures. While it would be somewhat easy to delete items in order to create what appears to be a valid scale, the need for attending to cultural constructs while preserving the theoretical integrity of the measure seems evident if a valid assessment suitable for research is to emerge. 2 Use of the Personal Report of Communication Anxiety (PRCA-24) in Japanese Contexts: Reliability, Structural Integrity, and Response Validity Introduction Communication anxiety has been an especially important variable in communication research in Japan (Klopf, 1984; Tanaka, 1996). One of the most prominently used measures of this trait has been the Personal Report of Communication Anxiety (PRCA-24), and previous studies have indicated consistently high reliability for translated versions of this scale (see McCroskey, Gudykunst, and Nishida, 1985; Pribyl, Keaten, Sakamoto, and Koshikawa, 1998). While these studies have supported the overall viability of using the PRCA-24 for research in communication, little had been done to assess the purported underlying dimensions of the scale or how well it reflects aspects of communication apprehension. In a Likert based scale such as the PRCA-24, it is necessary that the measure include a sufficient number of items necessary to incorporate cultural perceptions related to the underlying trait. In addition, the measure should exclude those aspects that are irrelevant and so preserve parsimony (John & Benet-Martínez, 2000; Judd & McClelland, 1998). With developments in structural equation modeling (Arbuckle & Wothke, 1999) and item response theory, most especially in the area of Rasch analysis (Wright, 1999; Wright & Masters, 1982), more complex issues of scale measurement may be addressed. Specifically, confirmatory factor analysis (CFA) allows assessment of presumed multidimensional structures, and Rasch analysis provides a means of determining how accurately, continuously, and completely items in a scale are measuring a purported continuous underlying trait. The primary purpose of this study was to assess a translated version of the PRCA-24 in terms of its structural integrity and its ability to provide a continuous and inclusive measure of trait anxiety. Personal Report of Communication Anxiety (PRCA-24) The PRCA-24 is a set of 24 items with Likert type responses that attempt to measure the latent trait of communication apprehension in four state situations: group, meeting, dyadic, and public (McCroskey 1970, 1982; Daly & McCroskey, 1984; Richmond & McCroskey, 1998). A number of studies have revealed strong content validity (McCroskey, 1978; McCroskey, Beatty, Kearney, & Plax, 1985) in the 3 original English version, and the PRCA-24 has been shown to correlate with most other measures of trait anxiety. Consistent measures of α ≥ .90 and item-total correlations ranging from .40 to .69 have been found with α for subscales greater than .75 (McCroskey, 1990, 1992). Using similar classical measures, strong reliability has also been reported for Japanese translations (see McCroskey, 1978; McCroskey, Gudykunst, & Nishida, 1985; Klopf, 1984). Despite its use in Japan, Levine & McCroskey (1990) indirectly questioned the validity of the PRCA24 with concerns that some items might be culturally bound. However, summed totals and subscales have continued to be used (McDowell, & Yotsuyanagi, 1996). Replication of structures with the four subscales intact has been somewhat elusive, even in the English version. Levine & McCroskey (1990) tested first and second order confirmatory factor analysis models and reported strong support for a second order model in which the latent trait of communication apprehension is reflected in the four subscale dimensions. Perhaps most important for this study, they failed to replicate such goodness of fit in the one non-United States culture sampled. This led them to question whether all items would be appropriate for research in cultures other than the United States. Pribyl, Keaten, Sakamoto, & Koshikawa (1998) were unable to replicate the four subscale factor structure in a Principle Components analysis of their Japanese version. While they did not report results of any confirmatory factor analysis or comparisons with other methods generally recommended in exploratory analysis (see Johnson & Wichern, 1998), their four-component rotation showed meeting items loading in various degrees on the public component. Despite its historical base in the theoretic development of the construct, they recommend removal of the public dimension from Japanese versions of the scale even though this component explained most variation in their model with public items having relatively strong and pure loadings. These researchers concluded that the present version of the PRCA-24 has doubtful use as a multidimensional measure of anxiety. Still, they indicated that a summation of all items should be used as a general measure of communication anxiety. However, they provided no real evidence for supporting the viability of using the PRCA-24 as a one-dimensional measure of trait anxiety. 4 Research Questions While the work mentioned in the previous section has assessed the reliability and validity of Japanese versions of the PRCA-24, rigorous testing of the internal structure and ability of the scale to discriminate along a continuum of trait anxiety has been lacking. No confirmatory analysis has been conducted on their theoretical structure of the scale, nor has any item response theory based assessment been made. Given the discussion and rationale above, the following specific questions were the focus of this research: 1. How reliable and internally consistent is a translated version of the PRCA-24? That is, can reliability be further replicated with Japanese subjects responding to a Japanese version of the scale? 2. How valid are the structural integrity and theoretic dimension or dimensions underling the purported trait measured by the PRCA-24? Are the underlying dimensions represented by the four subscales replicable in Japanese versions? 3. How viable is the PRCA-24 as a continuous measure of communication apprehension? Does the PRCA-24 provide a measure that adequately assesses the trait from low to high levels? The first research question includes analysis using classical measures of reliability. It is included to provide a basic check of the data and to replicate research that has included classic assessments made on Japanese versions. 5 Method The PRCA-24 was administered with three other anxiety instruments. 1 While some of the scales have been previously translated into Japanese, and some linguistic variability could affect comparisons with those translations, multiple translators working on all four measures were used in order to preserve internal consistency as much as possible. Three independent Japanese translators, including a professor of English, a graduate student in English-Japanese studies, and a professional EnglishJapanese translator were asked to translate the scales. Another professional English-Japanese translator did a back translation of the translated scales from Japanese into English. Following this work, all translations were compared, and unanimous agreement was reached by all translators on all items. The translated version of the PRCA-24 used in this study may be found in the Appendix. Scales were administered to 350 Japanese college students (78 males, 272 females) consisting primarily of freshmen, sophomores, and juniors at a Kanto area university. To avoid response bias, instruments were presented in four different order combinations using mutually orthogonal Latin Squares. Both Japanese and non-Japanese administered the instruments. Subjects were told that research was being conducted by a professor in the university with no mention of that professor’s cultural background. The measures, including all demographic items, were in Japanese. Items with negative wording were reverse scored. Univariate tests for outliers (above or below 3.0 standard deviations) and Mahalanobis tests for multivariate outliers were conducted. Extreme responses appeared consistent with having extreme levels of the trait variable, and no subject was consistently an outlier on more than one scale. Further, analysis was conducted both with and without outliers, and results were substantially the same. It was judged best to be conservative relative to throwing out any data and consequent information. No subjects were removed from the study. There were no missing values, and F-tests revealed no significant differences due to ordering of the instruments. 1 Other scales included Embarrassability (Modigliani, 1968; Miller, 1987), Fear of Negative Evaluation (Watson and Friend, 1969), and Interaction Anxiousness (Leary, 1983). A comparison and analysis of all four instruments is anticipated in a forthcoming report. 6 Analysis proceeded in three phases based on the research questions mentioned earlier. In order to provide replication for previously reported results (Research Question 1), classical item analysis was conducted on the data including calculation of alpha coefficients and item-total correlations (Carmines & Zeller, 1979). To evaluate construct validity in terms of structural integrity (Research Question 2), confirmatory factor analyses were then performed on the data. Attention was paid to the concerns raised by Levine & McCroskey (1990) relative to the validity of the PRCA-24 and subtests for use in cultures outside the United States. Hence, a first order model was initially used to assess this scale. A first-order model with the four subscales as correlated factors was then tested, and this was followed by a second order model with the four subscales as factors reflecting the latent trait of communication apprehension (see Figures 1 and 2 in the Results Section). Maximum Likelihood methods were used in all models, and a scale value of one was assigned to unobserved variables. With these parameters, all models were identified. Several measures were calculated and compared for goodness of fit, but the following were considered essential and reported in the results (see Kline, 1998; Arbuckle & Wothke, 1999; Byrne, 2001): 1. CMIN – Minimum Value of the Discrepancy (Brown, 1982). 2. CMIN/df – CMIN divided by degrees of freedom. Values over 3 indicate weakness in the model (Carmines & McIver, 1981; Marsh & Hocevar, 1985). 3. AGFI – Adjusted Goodness of Fit Index (Arbuckle & Wothke, 1999; Tanaka & Huba, 1985). Values above .95 are common in adequate models, and values below .90 indicate poorly fitting models. 4. CFI – Comparative Fit Index (Bentler, 1990). Values near one indicate good fit. Values below .90 indicate poor fit. 5. RMSEA – Root Mean Square Error of Approximation (Browne & Cudeck, 1993). In general, values of .05 or less indicate well fitting models; values above .05 and below .08 indicate usable but weak models; values greater than .10 indicate unusable models. Confirmatory factor analysis is normally conducted on models with strong theoretical bases and with the assumption that any exploratory factor analysis has been performed in the scale’s original 7 development. Such an assumption was made in this study relative to testing the integrity of constructs for the PRCA-24. Exploratory analyses and assessment of misfit indices of confirmatory factor analysis is generally not advised (see Cliff, 1983; Kline, 1998), and extensive analysis of scale revision is well beyond the bounds of this study. However, given failure of models to adequately fit theorized constructs, exploratory analysis was conducted to gain a sense of dimensionality and provide reference for assessment of errant items as an aid for future research. Consistent with the confirmatory technique used earlier, a principle factors approach was taken. Rotated factors were chosen based on theoretical assumptions outlined in original scale development. Given that underlying dimensions can and perhaps should be correlated because they theoretically measure one latent trait, an oblique (PROMAX) solution was derived (see Gorsuch, 1983; Maruyama, 1998; Johnson & Wichern, 1998; Tabachnick & Fidell, 2001). In order to assess the ability of the PRCA-24 to consistently and inclusively measure an underlying trait of communication apprehension, a one-parameter Rasch analysis was conducted on the data (Research Question 3). Developments in item response theory, and especially the application of Rasch methods, have provided a means of assessing the ability of psychological measures to provide valid and continuous measures of underlying traits (see discussions in Hambleton, Swaminathan, & Rogers, 1991; Fischer & Molenaar, 1995; Van der Linden & Hambleton, 1997; Embretson, & Reise, 2000). Further, while Rasch Analysis has generally been applied in educational testing, its applicability to assess measurement properties of rating based instruments, especially those that have Likert based responses, has been amply demonstrated (Wright, 1999; Wright & Masters, 1982; Bond & Fox, 2001). A full explication of the underlying assumptions and ensuing controversies in Rasch analysis is far beyond the scope of this article; readers unfamiliar with the basic tenets are urged to consult the sources cited. Rasch analysis was chosen in this study in order to determine how well items in the PRCA-24 cover a continuum of varying levels of trait anxiety among subjects responding to the scale. While the theoretical multi-dimensionality of the PRCA-24 would be a direct contradiction of the assumptions of Rasch modeling, this assessment provides a viable means of the testing the contentions of Pribyl, 8 Keaten, Sakamoto, & Koshikawa (1998) that summed items may provide a good overall measure of communication apprehension. SPSS was used for outlier, reliability, correlational, and factor analysis procedures. Confirmatory factor analysis was conducted using AMOS (Arbuckle, 1994; Arbuckle & Wothke, 1999). WinSteps (Linacre, 2001) was used for one-parameter Rasch analysis. 9 Results Internal Consistency - Reliability A summary of descriptive statistics and measures of reliability for Japanese subjects in this study may be seen in Table 1. The instrument and subscales showed strong α coefficients and overall high itemscale correlations. Measures of α for the individual subscales of the PRCA-24 ranged from .72 to .83. Though some item-total correlations were low, these results were consistent with earlier findings on both English and Japanese versions of the scales, thus indicating that such metrics could be replicated with this data (Research Question 1). Table 1. Summary of reliability and descriptive measures. .901 Lowest Item Scale Correlation .242 Highest Item Scale Correlation .676 Common Inter-Item Correlation .275 Public .812 .475 .651 .419 20.43 26.26 5.12 Group .798 .416 .684 .397 19.73 22.83 4.78 Meeting .830 .435 .647 .449 19.35 24.38 4.94 Dyadic .721 .394 .560 .301 16.47 18.90 4.35 Measure Cronbach’s PRCA-24 α 10 Mean Var SD 75.98 228.21 15.11 Structural Integrity – Confirmatory Factor Analysis A summary of measures for confirmatory factor analyses of the scale may be seen in Table 2. Models for the first order and second order factor structures involving the four subscales may be seen in Figures 1 and 2. Table 2. Summary of Confirmatory Factor Analysis measures Scale (Number of Components) PRCA-24 (1) CMIN df CMIN/df AGFI CFI RMSEA 1566.83 252 6.218 .633 .692 .122 PRCA-24 (4)* 11082.49 246 4.400 .738 .785 .099 PRCA-24 (4)** 1129.77 251 4.501 .736 .748 .100 * First Order ** Second Order Assessment revealed weaknesses in all models. All RMSEA terms were greater than .05. The PCLOSE statistic (Browne and Cudeck, 1993; Arbuckle and Wothke, 1999) tests the null hypothesis that the population RMSEA is not significantly higher than .05. This was less than .001 for all models, thus leading to rejection of that hypothesis for all. Relative to the findings of Levine and McCroskey (1990), both the first order and second order models of the PRCA-24 using four subscales as factors showed weak fit with this data. These results indicate that structural models either assumed or tested in earlier development of the scale could not be replicated in this study with confidence (Research Question 2). 11 Figure 1. First Order Confirmatory Factor Analysis – PRCA-24 with four subscales; Standardized estimates. .43 e1 PRCA01 e2 PRCA02 e3 PRCA03 .40 PRCA04 .46 .37 .65 .68 .61 e5 .63 .49 .24 .77 PRCA05.60 e6 PRCA06 e4 Groups .78 .28 e7 PRCA07 e8 PRCA08.51 e9 PRCA09 .53 PRCA10 e10 e11 e12 .40 .51 PRCA11.57 PRCA12 .53 .64 .71 .73 .71 .75 Meetings .55 .17 e13 PRCA13 e14 PRCA14 .32 e15 PRCA15.25 PRCA16 e16 e17 e18 .42 .16 PRCA17 .56 PRCA18 PRCA19 e20 PRCA20 e21 PRCA21 .39 PRCA22 e22 e23 e24 .26 .50 .56 PRCA23.27 PRCA24 .57 .41 .65 .56 .50 .82 Dyad .40 .75 .38 .60 e19 .40 .78 .51 .70 .62 .75 .52 12 Public Figure 2. Second Order Confirmatory Factor Analysis – PRCA-24 with four subscales; Standardized estimates. .36 e1 PRCA01 e2 PRCA02 e3 PRCA03 e4 PRCA04 e5 PRCA05 e6 PRCA06 .46 .37 .38 .24 .60 .60 .68 .61 R1 .62 .49 Groups .77 .63 .66 e12 PRCA12 e11 PRCA11 e10 PRCA10 e9 PRCA09 .75 .41 .64 Meetings PRCA08 .50 .78 e8 .50 .52 .56 .81 .71 R2 .72 .25 e7 .79 .88 PRCA07 CA .17 e13 PRCA13 .42 R3 .42 .65 e14 PRCA14 e15 PRCA15 .55 .25 .50 Dyad PRCA16 .42 .28 e16 .31 .17 e17 PRCA17 .74 .55 e18 PRCA18 R4 .56 e19 e20 e21 e22 e23 PRCA19 .25 PRCA20 .47 PRCA21 .39 PRCA22 .56 .75 .50 Public .69 .77 .63 .75 .52 PRCA23 .27 e24 PRCA24 13 .53 .88 Structural Integrity - Exploratory Factor Analysis A Bartlett’s χ 2 of 3665.855 (p<.001) and a KMO index of .878 were obtained for the data, thus indicating above criterion levels for both sampling and factorability. Results of Principle Axis factor analysis may be seen in Table 3. Relatively large discrepancies in communality estimates were found on all measures indicating lack of association for some items. In addition, multicollinearity indicated potential problems with redundancy. Table 3. Summary of Principle Axis factor analysis Components Extracted Variance Explained by Components Lowest Communality (Item) Highest Communality (Item) Determinant 4 45.830 .289 (Item 16) .685 (Item 9) 0.000 The four rotated factors for the PRCA-24 may be seen in Table 4. The first factor appears to have a mix of loadings relative to both meeting and public situations. The second factor seems most close to the group situation, and the third factor contains items related to dyads. The fourth factor includes items from two of the theoretic situations and appears to relate to communication with “new” people. 14 Table 4. Personal Report of Communication Anxiety - Structure Matrix Factor 1 2 3 4 PRCA19 Fear Giving Speech .754 .491 .250 .259 PRCA23 Prospect Speech Confidence .709 .432 .260 .175 PRCA12 Relaxed Questions Meetings .705 .573 .295 .215 PRCA10 Afraid Express Meetings .703 .531 .574 .114 PRCA21 Relaxed Giving Speech .699 .371 .054 .162 PRCA09 Calm, Relaxed Express Opinion Meeting .674 .554 .243 .206 PRCA11 Communicating Meetings Uncomfortable .658 .561 .524 .087 PRCA22 Thoughts Confused, Jumbled Speech .635 .373 .292 .155 PRCA24 Speech Nervous, Forget Facts .514 .265 .338 .142 PRCA20 Body Tense, Rigid Speech .499 .251 .184 .143 PRCA06 Calm, Relaxed Groups .453 .792 .171 .299 PRCA07 Participate Meetings .364 .686 .297 .313 PRCA02 Comfortable Groups .413 .662 .255 .214 PRCA01 Participating Groups .419 .652 .299 .095 PRCA04 Involved Groups .360 .630 .156 .058 PRCA08 Calm, Relaxed Meetings .525 .617 .271 .200 PRCA03 Tense, Nervous Groups .384 .605 .312 .297 PRCA18 Speak Up Conversation .309 .299 .751 .204 PRCA15 Tense, Nervous Conversations .161 .184 .649 .200 PRCA14 Fear Speaking Conversations .396 .310 .533 .261 PRCA16 Calm, Relaxed Conversations .128 .135 .526 .192 PRCA13 Conversation New Acquaintance .160 .214 .316 .691 PRCA17 Conversing New Acquaintance Relaxed .340 .287 .256 .647 PRCA05 Groups New People Tense, Nervous .364 .492 .239 .510 Response Validity - Rasch Analysis Results of goodness of fit for items in the Rasch model may be seen in Table 5. Looking at both infit and outfit measures, several scale items (note especially 6, 8, 11, 12, 13, 15, 16, 20, 24) show inadequate fit to a purported continuous measure of the underlying trait. 15 Table 5. Measures of Goodness of Fit to Rasch Model +------------------------------------------------------------------------+ |ENTRY RAW | INFIT | OUTFIT |SCORE| | |NUMBER SCORE COUNT MEASURE ERROR|MNSQ ZSTD|MNSQ ZSTD|CORR.| ITEMS | |------------------------------------+----------+----------+-----+-------| | 1 1062 347 .11 .06| .97 -.4|1.01 .2| .53| PRCA01| | 2 1084 347 .04 .06| .88 -1.9| .89 -1.7| .54| PRCA02| | 3 1121 347 -.07 .06|1.00 .1|1.02 .2| .53| PRCA03| | 4 1123 347 -.08 .06| .94 -.9| .99 -.1| .47| PRCA04| | 5 1255 347 -.51 .06|1.21 2.8|1.22 2.8| .50| PRCA05| | 6 1193 347 -.30 .06| .75 -3.9| .75 -3.9| .58| PRCA06| | 7 1125 347 -.08 .06| .86 -2.1| .87 -1.9| .55| PRCA07| | 8 1116 347 -.06 .06| .69 -5.1| .73 -4.2| .57| PRCA08| | 9 1264 347 -.54 .06| .87 -1.9| .91 -1.3| .59| PRCA09| | 10 991 347 .33 .06| .86 -2.2| .84 -2.4| .64| PRCA10| | 11 985 347 .34 .06| .77 -3.8| .77 -3.6| .63| PRCA11| | 12 1225 347 -.40 .06| .66 -5.5| .67 -5.2| .62| PRCA12| | 13 1163 347 -.20 .06|1.36 4.6|1.40 5.1| .36| PRCA13| | 14 1018 347 .24 .06|1.05 .8|1.07 1.0| .48| PRCA14| | 15 729 347 1.18 .06|1.30 3.8|1.50 5.6| .38| PRCA15| | 16 782 347 .99 .06|1.63 7.4|1.74 8.1| .32| PRCA16| | 17 1175 347 -.24 .06|1.01 .1|1.04 .6| .44| PRCA17| | 18 833 347 .82 .06| .98 -.3|1.00 .0| .47| PRCA18| | 19 1239 347 -.45 .06| .91 -1.4| .90 -1.4| .61| PRCA19| | 20 1114 347 -.05 .06|1.39 5.2|1.44 5.5| .44| PRCA20| | 21 1275 347 -.57 .06|1.06 .8|1.08 1.0| .51| PRCA21| | 22 1163 347 -.20 .06|1.01 .1|1.03 .5| .53| PRCA22| | 23 1188 347 -.28 .06| .90 -1.5| .88 -1.7| .57| PRCA23| | 24 1105 347 -.02 .06|1.20 2.7|1.25 3.4| .47| PRCA24| |------------------------------------+----------+----------+-----+-------| | MEAN 1097. 347. .00 .06|1.01 -.1|1.04 .3| | | | S.D. 143. 0. .45 .00| .23 3.2| .26 3.3| | | +------------------------------------------------------------------------+ A graphical comparison of Rasch score distributions for subjects (persons) and items is presented in Figure 3. Perusal of this comparison shows that the PRCA-24 is, overall, an “easy” measure in Rasch terms. That is, with the exception of some items on the dyadic dimension (15, 16, 18), full coverage of higher levels of the anxiety trait appears inadequate. Japanese respondents find it easy to agree with the scale’s items, i.e., easy to indicate high levels of apprehension. This corresponds with previous studies that have shown Japanese subjects to score consistently higher on the PRCA-24 than their counterparts in the U.S. (see again McCroskey, Gudykunst, & Nishida, 1985; Klopf, 1984). Weakness in the ability of the scale to adequately measure higher levels of the trait is further seen in Figure 4. Here, item scores are plotted by increasing levels of the underlying trait. 16 Figure 3. Comparison of Person and Item Rasch Scores of PRCA-24. 4 3 2 1 0 -1 -2 -3 EACH PERSONS MAP OF ITEMS <more>|<rare> . + | | | | | | | + | | | . | | . | | + | . | . | . T| ## | #### | .# | PRCA15 .### + PRCA16 .##### S|T PRCA18 .### | ###### | .###### |S ######### | PRCA10 .######### | PRCA14 .########## M| PRCA01 .########## +M PRCA02 .######## | PRCA03 ###### | PRCA06 .###### | PRCA12 .#### S|S PRCA05 .### | PRCA21 .### | . |T ## + # | . T| | # | . | | . | + | | # | | | | | . + <less>|<frequ> '#' IS 3. 17 PRCA11 PRCA08 PRCA04 PRCA13 PRCA20 PRCA07 PRCA17 PRCA09 PRCA19 PRCA24 PRCA22 PRCA23 Figure 4. Continuity of PRCA items in measuring underlying trait apprehension PRCA Items Sorted by Increasing Difficulty of Agreement 1.2 1 0.8 Difficulty 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 21 9 5 19 12 6 23 17 13 22 4 7 3 8 20 24 2 Item Number 18 1 14 10 11 18 16 15 Discussion Several conclusions relative to a translated version of the PRCA-24 seem apparent from the results of this study. First, while the translated scale shows internal consistency as measured by classical reliability statistics, further analysis indicates weakness in terms of structural integrity and the underlying theoretical assumption of four dimensions. Second, the ability of the scale to be used as an overall measure of anxiety must be questioned. The ability of the scale to adequately cover and discriminate higher levels of trait anxiety, i.e., the lack of items for which subjects find it more difficult to agree, appears to be a prime weakness of the scale. Obviously, results of this study indicate that care should be taken in using summed values of this scale for research or clinical assessment in Japan. Why does a Japanese version of the PRCA-24 show weakness in terms of clarity and coverage of theoretical and internal constructs related to the trait of communication apprehension? Translation or measurement anomaly cannot be eliminated as a source of error variation, but the high measures of reliability indicate that reasons may lie elsewhere. As noted earlier, a stable structure for this scale has been elusive, even in its English form. Thus, some aspects of model misfit in this study could be due to instabilities or inadequacies in the theoretical foundation of the construct itself. For example, the Dyadic dimension of the PRCA-24 shows lower item-total correlations as well as lower correlations in both the first and second order confirmatory factor analysis models (Figures 1 and 2). This subscale has also appeared to be a somewhat less stable factor than other subscales in prior research on the English version (McCroskey & Beatty, 1984; McCroskey, 1990, 1992; Levine and McCroskey, 1990). On the other hand, as seen in results of Rasch analysis, only items of this theoretic dimension appear to have any ability to measure higher levels of communication apprehension. Obviously, for Japanese subjects, augmentation of the scale to include constructs involving communication situations that involve lower anxiety, and thus are more difficult to agree with, appears to be necessary in order for the measure to adequately cover the a continuous range of the apprehension trait and sub-dimensions. Results of the exploratory factor analysis of the PRCA-24 parallel those of Pribyl, Keaten, Sakamoto, & Koshikawa (1998). As in their results, items related to meetings appear to load highly on the public 19 dimension (again see Table 4). Japanese terms for “meeting” can include aspects of both group and public situations. Some “meetings” in Japan frequently involve formal presentations by participants rather than discussion while other “meetings” involve strong levels of interpersonal contact. For example, a regular “meeting” of full faculty at a university may require presentation skills on par with a public speaking situation while what is still termed a “meeting” of a small committee may involve much more interpersonal and less formal interaction. The grouping of items 17, 13, and 5 imply that interacting with a “new” person or someone not within ones sphere of acquaintance may be the basis of a state situation in itself and transcend the PRCA based contexts in which such interaction occurs. Modification or construction of items reflecting these differences awaits not only further empirical research in these specific interactions but in development of appropriate descriptive terms. Pribyl, Keaten, Sakamoto, & Koshikawa (1998) have recommended discarding items related to the public dimension since this form of communication is more rare in Japan than in the United States. However, such seems rather drastic given the strong loading of items on this dimension and the fact that such removal would destroy the historical and theoretical base of the scale. Further, the Rasch analysis would indicate that if anything, items need to be added to the scale to adequately cover situations which lie somewhere between conversations on the dyadic level and communication situations involving more public performance. Overall, additions or different groupings of items that maintain or enhance the fundamental theoretical situations and also account for different types of communication in each such as performance, participation, conversation, or social adjustment (establishing appropriate communication patterns with a new acquaintance) may be appropriate. Clarification of the theoretical base for understanding various communication situations and terms describing these situations seems necessary in future development of the PRCA and similar anxiety measures for use in Japan. At present, a theoretical base for such is somewhat weak and lacking in strong empirical support. Objective and empirical research into various forms of anxiety and modes of communication to be measured in the PRCA may lead to further determination of appropriate modifications or additions of items in this still potentially valuable instrument. 20 Conclusion That future research with the PRCA-24 in its present form should be conducted with caution seems evident. However, though it may be easy to factor analyze, delete, or force items of the PRCA-24 onto a structure that has no clear theoretical base, such has been avoided in this study for obvious reasons. Instead, these results should provide a base for reevaluating the theoretic base of the PRCA-24 as it reflects communication apprehension in Japanese culture and should also aid in determining which items should be modified, deleted, or augmented in order to produce the most parsimonious, theoretically rich, and culturally independent measure. 21 Appendix このアンケートの答え方:この質問表は人とのコミュニケーションに関するあなたの気持ちを尋ねる 24の項目から構成されています。各項目の内容があなたにどの程度あてはまるのかを次の(1)ま ったくその通りだと思う、(2)その通りだと思う、(3)どちらとも言えない、(4)そうは思わ ない、(5)まったくそうは思わない、の中から選択し、各項目の前にある空欄に1から5までの数字 を記入して下さい。正しい答えや間違った答えといったものはありません。よく似た記述が繰り返さ れても心配しないで下さい。時間をかけずに取り組み、あなたの第一印象を記入して下さい。 1. 2. 3. 4. 5. 6. 7. 8. 集団討論に参加することが好きではない。 たいてい、集団討論に参加している時は居心地がよく落ち着いた気持ちでいられる。 集団討論に参加している時は緊張して神経を使う。 集団討論に取り組むことが好きだ。 新しい人たちと集団討論を行うと緊張して神経質になる。 集団討論に参加している時は落ち着いてリラックスしている。 たいてい、会議やミーティングに参加しなければならない時は神経を使う。 いつも、会議やミーティングに参加している時は落ち着いた気持ちでリラックスしてい る 。 9. 会議やミーティングで意見を述べるように求められた時、とても落ち着いた気持ちで リラックスしている。 10. 会議やミーティングで自分の意見を述べることが怖い。 11. 会議やミーティングでの意見交換はいつも居心地が悪く落ち着かない。 12. 会議やミーティングで質問に答える時、私はとてもリラックスしている。 13. 新しく知り合った人と会話する時はとても神経を使う。 14. 会話の途中で自分の意見を述べることにまったく怖さを感じない。 15. 普段、会話する時はとても緊張し神経を使う。 16. 普段、会話する時はとても落ち着いてリラックスしている。 17. 新しく知りあった人と会話する時はとてもリラックスしている 。 18. 会話をしている時に自分の意見を述べることに怖さを感じる。 19. スピーチをすることはまったく怖くない。 20. 21. 22. 23. スピーチをしている時、身体の一部分が緊張して硬直するように感じる。 スピーチをしている時、リラックスした気持ちでいる。 スピーチをしている時、頭の中が混乱し考えがまとまらない。 スピーチをしなければならない状況を前に、自信を持ってスピーチをする心の準備 ができている。 24. スピーチをしている時、あまりにも神経が高ぶり、よく知っていることでさえも忘 れてしまう。 The following items are reversed: 2, 4, 6, 8, 9, 12, 14, 16, 17, 19, 21, 23 Subscales are calculated from raw scores as follows (instrument item number in parenthesis): Group = 18 - (1) + (2) - (3) + (4) - (5) + (6) Meeting = 18 - (7) + (8) - (9) + (10) - (11) + (12) Dyadic = 18 – (13) + (14) – (15) + (16) – (17) + (18) Public = 18 + (19) – (20) + (21) – (22) + (23) – (24) Overall = Group + Meeting + Dyadic + Public 22 References Arbuckle, J.L. (1994) AMOS: Analysis of moment structures. Psychometrika, 59, 135-137. Arbuckle, J. L. & Wothke, W. (1999) Amos 4.0 user’s guide. Chicago: SmallWaters Corporation. Bentler, P. M. (1990) Comparative fit indexes in structural models. Psychological Bulletin, 107, 238246. Bond, T. G. & Fox, C. M. (2001) Applying the Rasch Model. London: Lawrence Erlbaum Associates. Browne, M. W. (1982) Covariance structures. In D. M. Hawkins (Ed.) Topics in applied multivariate analysis (pp. 72-141). Cambridge: Cambridge University Press. Browne, M. W. & Cudeck, R. (1993) Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.) Testing structural equation models (pp. 136-162). Newberry Park, CA: Sage. Byrne, B. M. (2001) Structural equation modeling with AMOS. London: Lawrence Erlbaum Associates. Carmines, E. G. & McIver, J. P. (1981) Analyzing models with unobserved variables. In G. W. Bohrnstedt & E. F. Borgatta (Eds.), Social measurement; Current issues. London: Sage. Carmines, E. G. & Zeller, R. A. (1979) Reliability and validity assessment. London: Sage Publications. Cliff, N. (1983) Some cautions concerning the application of causal modeling methods. Multivariate Behavioral Research, 18, 115-126. Daly, J. A. & McCroskey, J. C. (Eds.) (1984) Avoiding communication: Shyness, reticence, and communication apprehension. Beverly Hills, CA: Sage Embretson, S. E. & Reise, S. P. (2000) Item Response Theory for psychologists. London: Lawrence Erlbaum Associates. Fischer, G. H. & Molenaar, I. W. (Eds.) (1995) Rasch Models: Foundations, recent developments, and applications. New York: Springer-Verlag. Gorsuch, R. L. (1983) Factor analysis. Hillsdale, NJ: Lawrence Erlbaum Associates. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991) Fundamentals of Item Response Theory. London: Sage Publications. John, O. P. & Benet-Martínez, V. (2000) Measurement: reliability, construct validation, and scale construction. In H. T. Reis & C. M. Judd (Eds.) Handbook of research methods in social and personality psychology (pp. 339-369). Cambridge: Cambridge University Press. Johnson, R. A. & Wichern, D. W. (1998) Applied multivariate statistical analysis, Upper Saddle River, NJ: Prentice-Hall. Judd, C. M. & McClelland, G. H. (1998) Measurement. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.) The handbook of social psychology (4th Ed., Vol. 1, pp. 180-232). Boston: McGrawHill. 23 Kline, R. B. (1998) Principles and practice of structural equation modeling. New York: The Guilford Press. Klopf, D.W. (1984). Cross-cultural apprehension research: A summary of Pacific Basin studies. In J. A. Daly & J. C. McCroskey (Eds.), Avoiding communication: Shyness, reticence, and communication apprehension (pp. 157-172). Beverly Hills, CA: Sage. Kondo, S. (2001) The perceived effectiveness of anxiety coping strategies in social situations. Human Communication Studies, 29, 1-9. Leary, M. R. (1983) Social Anxiousness: The Construct and Its Measurement. Journal of Personality Assessment, 47, 66-75. Leary, M. R. (1991) Social anxiety, shyness, and related constructs. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 161194). New York: Academic Press. Leary, M. R. & Kowalski, R. M. (1987) Manual for the Interaction Anxiousness Scale. Social and Behavioral Sciences Documents, 16, 2, Manuscript No. 2774. Leary, M. R. & Kowalski, R. M. (1993). The Interaction Anxiousness Scale: Construct and criterionrelated validity. Journal of Personality Assessment, 61, 136-146. Levine, T. R., & McCroskey, J. C. (1990). Measuring trait communication apprehension: A test of rival measurement models of the PRCA-24. Communication Monographs, 57, 62-72. Linacre, J. M. (2001) A user’s guide to WINSTEPS. Chicago: MESA Press. Marsh, H. W. & Hocevar, D. (1985) Application of confirmatory factor analysis to the study of selfconcept; First and higher order factor models and their invariance across groups. Psychological Bulletin, 97, 562-582. Maruyama, G. M. (1998) Basics of structural equation modeling. London: Sage Publications. McCroskey, J. C. (1970) Measures of communication-bound anxiety. Speech Monographs, 37, 269277. McCroskey, J.C. (1978). Validity of the PRCA as an index of oral communication apprehension. Communication Monographs, 45, 192-203. McCroskey, J. C. (4th Ed.). (1982) An introduction to rhetorical communication, Englewood Cliffs, New Jersey: Prentice-Hall. McCroskey, J. C. (1992) Reliability and validity of the Willingness to Communicate Scale. Communication Quarterly, 40, 16-25. McCroskey, J. C., & Beatty, M. J. (1984) Communication apprehension and accumulated communication state anxiety experiences: A research note. Communication Monographs, 51, 79-84. McCroskey, J. C., Beatty, M. J., Kearney, P., & Plax, T. G. (1985) The content validity of the PRCA24 as a measure of communication apprehension across communication contexts. Communication Quarterly, 33, 165-173. 24 McCroskey, J.M., Gudykunst, W.B., & Nishida, T. (1985). Communication apprehension among Japanese students in native and second language. Communication Research Reports, 2, 11-15. McDowell, E. E. and Yotsuyanagi, N. (1996) An Exploratory Study of Communication Apprehension, Willingness To Communicate, and Sense of Humor between College Students from the United States and Japan. (ERIC Document Reproduction Service No. ED396349) Miller, R. S. (1987) The Nature of Embarrassability; Correlates and Sex Differences. Unpublished Manuscript, Sam Houston State University, Huntsville, Texas. Modigliani, A. (1968) Embarrassment and Embarrassability. Sociometry, 31, 313-326. Pribyl, C. B., Keaten, J. A., Sakamoto, M., & Koshikawa, F. (1998) Assessing the cross-cultural content validity of the Personal Report of Communication Apprehension Scale (PRCA-24). Japanese Psychological Research, 40, 47-53. Richmond, V. P. & McCroskey, J. C. (5th Ed.). (1998) Communication apprehension, avoidance, and effectiveness. Boston: Allyn and Bacon. Tabachnick, B. G. & Fidell, L. S. (2001) Using multivariate statistics, Needham Heights, MA: Allyn and Bacon. Tanaka, J. S. & Huba, G. J. (1985) A fit index for covariance structure models under arbitrary GLS estimation. British Journal of Mathematical and Statistical Psychology, 38, 197-201. Van der Linden, W. J. & Hambleton, R. K. (Eds.) (1997) Handbook of modern Item Response Theory. New York: Springer-Verlag. Watson, D. & Friend, R. (1969) Measurement of social evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448-457. Wright, B. D. (1999) Fundamental measurement for psychology. In S. E. Embretson and S. L. Hershberger (Eds.) New Rules of Measurement; What Every Psychologist and Educator Should Know (pp. 65-104). London: Lawrence Erlbaum Associates. Wright, B. D. & G. N. Masters (1982) Rating scale analysis; Rasch measurement. Chicago: MESA Press. 25
© Copyright 2024 Paperzz