Consulting Psychology Journal: Practice and Research
2011, Vol. 63, No. 4, 203–218
© 2011 American Psychological Association
1065-9293/11/$12.00 DOI: 10.1037/a0026537
EVIDENCE FOR THE VALIDITY OF 360
DIMENSIONS IN THE PRESENCE OF
RATER-SOURCE FACTORS
Nigel Guenole, Tony Cockerill, Tomas Chamorro-Premuzic, and
Luke Smillie
Goldsmiths, University of London
Empirical research on the structure of 360-degree feedback ratings indicates that the
source of the ratings (e.g., superiors, peers, subordinates) explains more variance than do
the performance dimensions or competencies being measured. One alarming implication
of this finding from studies of the internal validity of 360 ratings is that there appears to
be little evidence to support the common practice of interpreting 360s in terms of
dimension scores. To address whether rater-source factors are so pronounced that they
should preclude the use of dimension scores, we considered the question from an
external validity perspective and developed and tested a personality-based nomological
network around both dimension and rater-source factors in a 360 data set. Using a
sample of 825 managers and their feedback providers (3,300 participants overall), we
found that ratee personality correlated more strongly with dimension scores than with
source factors. This provides evidence to support the common practice of interpreting
360-degree feedback in terms of scores for separate dimensions and competencies,
despite most of the variance in observed ratings being due to rater-source factors rather
than dimensions.
Keywords: Multisource, 360-degree feedback, personality, dimensions, source factors
Multisource, or 360-degree, feedback is a popular option for organizations wanting to implement
performance assessment systems (Borman, 1997; Brett & Atwater, 2001). In usual implementations
of this approach, multiple raters—typically selves, superiors, peers, and direct reports/
subordinates—rate employee performance on multiple performance dimensions or competencies.
Broadly speaking, scores are interpreted in one of two ways. Under the first approach, agreement
between self-ratings and the ratings of others on targeted dimensions is examined, and the value of
differences between self- and other’s ratings in dimension ratings is considered a measure of
self-awareness (e.g., Church, 1997; Kulas & Finkelstein, 2007). Evidence suggests that selfawareness has implications for performance (Bass & Yammarino, 1991; Fletcher, 1997; Furnham &
Stringfield, 1994; Yammarino & Atwater, 1997), arguably because individuals who are highly
self-aware pay attention to feedback, which enables them to have a more accurate picture of their
Nigel Guenole, Tony Cockerill, Tomas Chamorro-Premuzic, and Luke Smillie, Psychology Department,
Goldsmiths, University of London, London, England.
Correspondence concerning this article should be addressed to Nigel Guenole, Psychology Department,
Goldsmiths, University of London, New Cross, SE14 6NW United Kingdom. E-mail: [email protected]
203
204
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
strengths and weaknesses (Fletcher, 1999; London & Smither, 1995; Wohlers & London, 1989;
Yammarino & Atwater, 1993). Taking the second approach, item scores are aggregated to form
dimension scores, either within or across rater groups. These scores are used for feedback or are used
in decision-making and validation studies (e.g., Atwater & Yammarino, 1992). Sometimes, dimension scores are compared to relevant norm groups to facilitate score interpretation (Morgeson,
Mumford, & Campion, 2005).
In either case, the data that result from crossing multiple raters with multiple dimensions present
psychologists with an opportunity to examine the validity of the rating instrument using a multitraitmultimethod (MTMM) design (Campbell & Fiske, 1959). For evidence of the construct-related
validity of a 360-degree feedback instrument under the MTMM framework, correlations between
ratings of different dimensions made by the same rater group (i.e., the mono-method-hetero-trait
correlations) ought to be smaller than the correlations between the same dimensions rated by
different observer groups (i.e., the hetero-method-mono-trait) correlations. However, this pattern
rarely occurs; more often, correlations among different dimensions within a rating group are higher
than correlations across rater groups on the same dimension (Lance, Hoffman, Baranik, & Gentry,
2008). Researchers such as Widaman (1985) and Eid, Lischetzke, and Nussbeck (2006) have
suggested that confirmatory factor analysis (CFA) ought to be used when analyzing MTMM data,
as it overcomes several limitations inherent to approaches based on simple inspection of correlations. CFA allows formal evaluation of the fit of the MTMM model to the data, permits separation
of systematic trait (hereafter: “dimension”) and method (hereafter: “source”) effects from nonsystematic measurement error, and allows dimension and rater-source factors to be linked to other
variables in a broader nomological network (Eid et al., 2006). However, findings from CFA
approaches to MTMM analyses of 360-degree feedback data also show stronger evidence of source
factors than dimensions. This is evident from analyses of the variance components of observed item
scores, where item factor loadings on source factors tend to be greater than loadings on the intended
dimensions1 (Conway, 1996; Hoffman, Lance, Bynum, & Gentry, 2010; Lance et al., 2008).
The lack of evidence for dimensions is troubling for practitioners, as it implies that employees
are receiving feedback on performance dimensions that might not exist or that were poorly
measured. When this occurs, employees could expend effort altering behavior that does not need to
be changed, which is ethically undesirable and economically inefficient. In such cases, it may be
more meaningful to ignore dimensions altogether and instead provide a single overall performance
score for the person assessed from the perspective of each rater group. One way to decide whether
to afford any relevance to dimensions in the presence of strong rater-source factors, and to decide
whether practitioners ought to adopt overall rater source scores, is to examine their respective
nomological networks. In other words, we could investigate the psychological meaningfulness of
dimensions and source factors by studying the ways they relate to external variables (Cronbach &
Meehl, 1955). This is the goal of this article.
Specifically, we wish to extend research on personality as a correlate of dimension and source
factors in 360-degree feedback by creating and testing a personality-based (i.e., the ratee’s personality) nomological network for dimension and source factors. In doing so, we answer the question
of whether rater-source factors in 360-degree feedback are so strong that they should preclude the
interpretation of dimension scores. Practically, this would mean only using total scores from each
rater group for organizational decision making and development planning. Personality is an
appropriate construct for inclusion in our nomological network of dimensions and, perhaps, source
1
Source factors are not the same thing as a rater group’s unique perspective on dimensions. Rather, source
factors represent variance not attributable to candidates’ standing on the dimension, and not attributable to
measurement error, but associated with each rater group’s perspective of candidates. Source factors can, in
principle, be minimized while allowing different rater groups to have different perspectives on dimensions.
While some people refer to source factors as “source effects,” we forego this terminology in favor of “source
factors,” which is more consistent with how this variance is modeled and avoids the term being misconstrued
as a rater groups’ unique perspective on the dimension of interest.
EVIDENCE FOR 360 DIMENSIONS
205
factors in 360-degree feedback research, because it is known to impact job performance through
intervening motivational mechanisms (Hogan & Holland, 2003; Hogan & Shelton, 1998).
The approach of using data external to 360-degree feedback instruments to validate their outputs
is consistent with an emerging chorus of consensus suggesting that internal validity evidence is an
incomplete basis for assessing the construct validity of 360-degree feedback (Borman, 1997; Lance
et al., 2008; Woehr, Sheehan, & Bennett, 2005). However, this research extends recent work by
these researchers in important ways. For example, in the most comprehensive examination of this
topic to date, Hoffman and Woehr (2009) studied narrow personality facets from the California
Psychological Inventory (CPI) as predictors of dimension and source factors. However, researchers
have yet to examine the efficacy of broad factors of personality, such as the Big Five, in the
prediction of dimension and source factors. Importantly, no study, to date, has presented a
comparison of the personality correlates of dimensions and the source factors derived from the
ratings of all typical rater groups (i.e., self, superior, peer, direct report). We address these points in
the following study.
Dimension and Source Factors
If the weaker-than-expected dimension variance in 360-degree feedback is a problem for the
interpretation of dimension scores in applied settings, it ought to be manifested in dimension scores
that do not systematically relate to theoretically relevant external variables. Alternatively, if
dimension scores from 360 ratings are psychometrically viable constructs for applied use, they
would relate, in meaningful ways, to external variables. Taking this line of reasoning further, to the
extent that source factors represent measurement error, they should not be expected to correlate with
external variables. If, on the other hand, source factors represent something substantive, then they
should correlate with external variables.
Hoffman and Woehr (2009) indicated that most studies, to date, that have examined the
nomological network of 360 ratings have focused on dimensions to the exclusion of source factors
and have found moderate evidence for their validity. The few exceptions to this trend that look at
the nomology of source factors indicate some validity for the ecological perspective on source
factors, suggesting that source factors are substantively meaningful (Hoffman et al., 2010; Hoffman
& Woehr, 2009; Lance, Teachout, & Donnelly, 1992; Lance, Baxter, & Mahan, 2006).
In this article, we do not argue for or against the perspective that source factors represent valid
components of the job performance criterion space. Rather, we examine the issue of whether source
factor variance is so pronounced that it invalidates interpretation of dimension scores. This is a
pivotal issue. If it is the case that dimensions are more strongly correlated with personality
dimensions than source factors are, regardless of whether there is more internal evidence of source
factors and whether they, too, correlate with personality, it provides empirical support for the
common practice of interpreting 360 results in terms of scores for different dimensions or competencies. Therefore, we studied whether dimensions or source factors correlate most strongly with
personality.
In the following study, we outline how we derived Big Five personality scores from a
proprietary personality instrument. This is appropriate given that the Big Five is the most widely
accepted personality model today (e.g., Barrick & Mount, 1991). We use a proprietary competency
model in the form that it is currently used in applied settings. This is because there is no widely held
consensus regarding the precise dimensionality of job performance, beyond perhaps suggesting that
there is a general factor in job performance ratings (Scullen, Mount, & Goff, 2000; Viswesvaran,
Schmidt, & Ones, 2005). Of course, useful taxonomic distinctions exist, for example, between
initiating structure and consideration (Fleishman & Harris, 1962) and between task performance and
contextual performance (Borman & Motowidlo, 1993). To provide such broad-brush feedback about
overall performance levels, however, or even the way one deals with tasks and people, would not
be detailed enough for most development applications. However, the more detailed dimensional
models offered, to date, by researchers such as Fleishman, Mumford, Zaccaro, and Levin (1991),
Tett, Guterman, Bleier, and Murphy (2000), or Bartram (2005) are rationally derived structures. In
206
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
the absence of factor analytic support for these models, we proceeded by adopting the intended
dimensionality of the competency model and reporting tests of its appropriateness from our
analyses.
Personality as an External Correlate of Dimensions and Source Factors
The correlations that Hoffman and Woehr (2009) presented among narrow CPI scales, broad
performance dimensions, and source factors were, in general, small to moderate. But correlations
between predictors and outcomes are thought to be larger when the breadth of the predictor matches
the breadth of the criterion (Ones & Viswesvaran, 1996; Paunonen, Rothstein, & Jackson, 1999;
Tett, Steele, & Beauregard, 2003). It is possible, then, that broader personality scales—for example,
measures of the Big Five—might correlate more strongly with dimension and source factors, even
though specific personality facets in the Hoffman and Woehr (2009) study did not.
To form hypotheses about the relationship between the Big Five and our 360-degree feedback
instrument, we drew on the work of Lievens, Chasteen, Day, and Christiansen (2006). Lievens et al.
(2006) used trait activation theory to identify which performance dimensions were related to
different Big Five dimensions. Trait activation theory suggests that job dimensions that prime
similar Big Five personality traits will be more highly correlated than those that do not. Using
subject matter experts, they identified which Big Five dimensions were likely to be most strongly
related to each of the performance dimensions of a taxonomy proposed by Arthur, Day, McNelly,
and Edens (2003). We capitalized on this work in the following way.
First, we classified each of the dimensions of the proprietary model (see Table 1) into the
taxonomy of Lievens et al. (2006), which, in turn, was based on Arthur et al. (2003) There was
unanimous agreement between two authors of this article, who both have PhDs in IndustrialOrganizational (I-O) psychology and over 30 years of combined work experience in personality and
job performance measures. We then examined whether dimensions were correlated with the Big
Five personality traits suggested by their classification into the Lievens et al. (2006) framework. The
hypothesized pattern of correlations between personality and 360-degree feedback dimensions is
Table 1
Dimension Descriptions for Competency Model
Dimension
Code
Information Search
IS
Concept Formation
CF
Conceptual Flexibility
CX
Empathy
Teamwork
EM
TW
Developing People
DP
Influence
IN
Building Confidence
Presentation
BC
PR
Proactivity
Continuous Improvement
PO
CI
Customer Focus
CuF
Description
Gathering a rich variety of information from many different sources
about events
Linking information to form new ideas that explain the underlying causes
of events
Seeing issues from many different perspectives to compare options prior
to taking action
Encouraging others to express openly their real thoughts and feelings
Creating effective teams within the unit and across related departments or
functions
Providing staff with the resources, coaching, feedback, and training to
develop their capability
Using persuasive arguments and the goals and interests of others to build
support for ideas
Making your stance on issues clear
Making clear and concise presentations and establishing effective
communication processes
Designing implementation plans and outlining actions and responsibilities
Setting goals and targets, and monitoring progress, in order to improve
performance
Setting targets focused on adding value for the customer
EVIDENCE FOR 360 DIMENSIONS
207
presented in Table 2. The table shows 12 hypothesized correlations between personality and
dimensions: Extraversion with Presentation; Empathy with Teamwork, Influence, Building Confidence; Agreeableness with Empathy and Teamwork; Conscientiousness with Proactivity and Continuous Improvement; and Openness with Information Search, Concept Formation, and Conceptual
Flexibility. We made no hypotheses about the correlations between the Big Five and source factors
because there is currently no theoretical basis suggesting that an aspect of a rating target’s
performance should be systematically related to an idiosyncratic property of the rating source. For
a discussion of this point, see Hoffman and Woehr (2009).
Method
Participants
Participants were 825 managers from a wide variety of white-collar occupations in banking,
pharmaceuticals, manufacturing, and transport industries in the United Kingdom. Participants were
two-thirds male; age and ethnicity were not recorded. Participants were of varying levels of
supervisory experience, though all were responsible for the performance of other staff or thought to
be capable of moving into such roles. Scores were used solely for personal development planning
and not for personnel-related decisions. In addition to participants completing a self-assessment of
personality and performance, their performance was also rated by one superior manager (hereafter,
“manager”), one peer, and one direct report (i.e., 825 participants ⫻ 4 raters ⫽ 3,300 sets of ratings
in total). It is, of course, common in 360-degree feedback to have multiple raters from selected rater
groups, but the data set we had access to, and to which we could match personality data, had just
one per rater group. It is also worth noting that for certain rater groups, there is never more than one
rater (e.g., “self”) and rarely more than one for (e.g., superior).
Measures
We used a proprietary 360-degree feedback instrument that assesses 12 competencies. Each
competency is associated with one of four rationally derived and conceptually distinct clusters:
thinking, developing others, inspiring, or achieving. A list of brief definitions of each of the 12
competencies is presented in Table 1. This model emerged from a qualitative review of the research
literature relating to effective managerial behavior, including key research at Ohio State University
(Stogdill, 1950), the University of Michigan (Likert, 1961), and Harvard University (Bales, 1950).
Table 2
Hypothesized Big Five Links With Dimension Factor Scores Based on
Lievens et al. (2006)
Arthur et al. (2003)
Communication
Consideration
Drive
Influencing Others
Organizing and Planning
Problem Solving
Tolerance for Stress & Uncertainty
Other
a
Corresponding dimension
Lievens et al. (2006) trait link
Presentation
Empathy, teamwork
Proactivity
Influence, Building Confidence
Continuous Improvement
Information Search, Concept
Formation, Conceptual
Flexibility
Not assessed
Not applicable
Extraversion
Agreeableness and Extraversiona
Conscientiousness
Extraversion
Conscientiousness
Openness
Emotional Stability
Not applicable
While agreeableness was noted as the primary trait linkage for empathy and teamwork from Lievens et al.
(2006), extraversion was expected to correlate based on consensus of the all authors. Extraversion was also a
strong second correlate of consideration type dimensions in the Lievens et al. (2006) study; hence, in our
analyses, we examine both agreeableness and extraversion as correlates of empathy and teamwork.
208
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
Each dimension was measured on a Likert-type scale by five items rated from 1 (strongly disagree)
to 6 (strongly agree). The average internal consistency estimate for each dimension within each rater
group is in excess of .7. For the purposes of our CFA analyses, we tested four theoretical models
covering three competencies each, one for each of the four clusters. We assessed each dimension
with two well-performing items, chosen on the basis of factor loadings from within rater-group
exploratory factor analyses. Hence, in the models we discuss here, each CFA model included 24
items (2 items per dimension ⫻ 3 dimensions ⫻ 4 rater groups). Two items per factor were selected
because multiple indicator MTMM models, which have the ability to separate out systematic error
from method specific trait variance, require a minimum of two indicators per dimension for each
rater group and for each measurement method. More than two indicators per factor per rater group
quickly become unwieldy in complex models, such as multiple-indicator CFA MTMMs, without
resorting to item parceling.
Big Five personality scores were derived for each participant from an item pool of personality
questions that comprise the proprietary personality questionnaire. To produce Big Five scores for
participants in our sample, we applied a scoring key to a subset of items that was specifically
identified using an independent sample of 122 managers to measure the Big Five. This involved the
independent sample of 122 leaders completing both the proprietary personality instrument and
Goldberg’s (1999) Big Five factor marker scales. Each item from the proprietary instrument was
then grouped by subject matter experts into a Big Five domain based on item content, and items
were selected from within each of the Big Five item pools to maximize the correlation with the
relevant Goldberg marker score. The reliability of each of the proprietary Big Five scales in the
independent sample and current sample was as follows: Extraversion (.88/.87), Agreeableness
(.83/.60), Conscientiousness (.80/.81), Emotional Stability (.86/.70), and Openness (.71/.71). The
uncorrected correlations between the proprietary Big Five scales and the corresponding Goldberg
Big Five marker was as follows: Extraversion (.80), Agreeableness (.66), Conscientiousness (.69),
Emotional Stability (.76), and Openness (.72). Correcting for measurement error (i.e., less-thanperfect reliability), the correlations between the proprietary Big Five scales and the corresponding
Goldberg Big Five marker were as follows: Extraversion (.91), Agreeableness (.94), Conscientiousness (.85), Emotional Stability (.97), and Openness (1.0). These results indicate that once the effects
of measurement error are taken into account, these two distinct measures of the FFM are tapping the
same constructs. The correlations among the five proprietary Big Five scales in this sample were all
moderate (i.e., lower than .4 in absolute value), which is a range similar to published intercorrelations of the Big Five personality factors (e.g., Guenole & Chernyshenko, 2005). Further information
on this instrument, including factor analytic support, is available from the authors.
Results
Our CFA approach involved fitting a series of multiple-indicator MTMM CFA models and selecting
the model that most closely fit the data. Importantly, in contrast to much of the previous research,
our study used item-level modeling (i.e., no parceling) and was based on second-order MTMM
models. We used MPlus version 6 with the WLSMV estimator (Muthén & Muthén, 2006). We
estimated the models as outlined by Eid et al. (2006) for multiple indicator models. The models
applied to each cluster included (a) a correlated dimensions model with three dimension factors and
no source factors; (b) a correlated exercises model that had four source factors and no dimension
factors; (c) a correlated uniqueness model that had a factor for each dimension and, in addition,
allowed correlations among residuals for ratings from the same source; (d) a correlated dimensions,
uncorrelated source model that specifies a measurement model for both dimensions and sources; and
(e) a correlated dimensions, correlated sources model. We then estimated correlations between Big
Five personality scores and dimension and source factor scores from the most appropriate MTMM
CFA model.
The model that best represented the data across three of the four clusters of leadership behaviors
(i.e., Thinking, Developing, Inspiring) was the correlated dimensions, correlated source factor
model. Figure 1 illustrates this model for the Thinking cluster. For the Achieving cluster, the best
EVIDENCE FOR 360 DIMENSIONS
209
Figure 1. Second-order MTMM model for the thinking cluster. Dimensions and source factors are
mutually uncorrelated; factor covariances and residuals omitted for clarity. IS, CF, CX represent
Information Search, Concept Formation and Conceptual Flexibility; S, M, C, and R stand for self,
manager/superior, peer, and direct report.
model was a correlated dimensions, partially correlated source factors model, where the self-source
factor was correlated with superior, peer, and direct report source factors, but superior, peers, and
direct report source factors were not correlated with each other. This final model, which was not
expected a priori, would benefit from further examination in a new data set. Fit statistics for these
four models, presented in Table 3, demonstrated strong evidence of model appropriateness, with two
of the four models passing the chi-square test of perfect fit at p ⬍ .01, one passing at p ⬍ .0001,
and the remaining model showing close-fit statistics that well exceed conventional guidelines for
close model– data approximation (e.g., Hu & Bentler, 1999).
210
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
Table 3
Fit Statistics for Second-Order MTMM Models
␹2
Thinking
CT
CM
CT-UM
CT-PCM
CT-CM
Developing
CT
CM
CT-UM
CT-PCM
CT-CM
Inspiring
CT
CM
CT-UM
CT-PCM
CT-CM
Achieving
CT
CM
CT-UM
CT-PCM
CT-CM
df
p
CFI
TLI
RMSEA
Did not converge
176.360
103
158.563
84
147.197
101
132.906
101
.000
.000
.002
.018
.986
.986
.991
.994
.989
.987
.993
1.000
.029
.033
.024
.020
Did not converge
248.206
112
165.680
98
147.440
107
139.084
110
.969
.000
.006
.032
.975
.984
.991
.993
.969
.986
.992
.995
.975
.029
.021
.018
Did not converge
265.014
122
167.677
110
173.487
118
176.513
117
.000
.000
.001
.000
.964
.985
.986
.985
.973
.988
.989
.988
.038
.025
.024
.025
.000
.964
.974
.039
.000
.978
.984
.031
Did not converge
254.860
114
Did not converge
196.340
111
Did not converge
Note. CFI ⫽ confirmatory fit index; CM ⫽ correlated methods; CT ⫽ correlated traits; MTMM ⫽ multitraitmultimethod; PCM ⫽ partially correlated methods; RMSEA ⫽ root mean square error of approximation; TLI ⫽
Tucker Lewis Index; UM ⫽ uncorrelated methods.
The original 12-dimension and four-cluster structure of the proprietary competency model was
rationally derived, as other performance models alluded to earlier. The process followed in its
development is described in Cockerill (1989), and factor analytic support is presented in Guenole,
Chernyshenko, Stark, Cockerill, and Drasgow (in press). It is important, given its rational conceptual
basis, to show empirically that analyzing the model in terms of its four-cluster structure is
appropriate. In addition to the well-fitting CFA models, the appropriateness of the taxonomy is
supported because the average within-cluster correlation in this study was .68, while the mean
correlation among dimensions between clusters was .36.
Subsequent analyses of the variance components of the second-order dimension- and
method-item indicators, which are listed in the Appendix, showed stronger evidence of source
factors than dimensions. This is evident from the factor loadings for the first-order factors,
representing a rater group’s assessment of a candidate for a particular behavior on the
second-order dimension and source factors. These loadings show that, by and large, the
second-order loadings on the source factors are greater than those on dimension factors. This
finding is consistent with the literature to date (Lance et al., 2008). But there is currently only
limited research answering the question of whether source factors or dimensions more strongly
correlate with broad personality, which is the focus of the current study. To answer this
question, Table 4 presents Big Five correlations with the dimension and source factors from the
Thinking, Developing, Inspiring, and Achieving clusters. We interpret all correlations keeping
in mind that the internal consistency reliability of the agreeableness personality dimension is
EVIDENCE FOR 360 DIMENSIONS
211
Table 4
Big Five Correlations With Latent Source and Dimension Factors
Thinking
IS
CF
CX
Self
Superior
Peer
Direct report
Developing
EM
TW
DP
Self
Superior
Peer
Direct report
Inspiring
IN
BC
PO
Self
Superior
Peer
Direct report
Achieving
PO
CI
CU
Self
Superior
Peer
Direct report
E
A
C
ES
O
⫺.04
⫺.09
⫺.06
.15
⫺.12
⫺.05
.06
.07
.00
⫺.03
.16
⫺.06
⫺.02
.05
.17
.08
⫺.03
.14
⫺.02
.00
.06
.01
.04
.07
.18
⫺.01
.09
.05
.00
⫺.03
.00
.06
.02
.04
⫺.02
.19
.22
.17
.16
⫺.09
⫺.03
⫺.05
.26
.30
.34
.17
⫺.09
.05
⫺.02
.02
.02
.02
.09
⫺.10
.00
.04
.14
.12
.12
.14
⫺.06
.05
⫺.07
.11
.07
.11
.07
⫺.01
.02
⫺.05
.28
.27
.29
.26
⫺.14
.05
.03
.21
.12
.22
.17
⫺.12
.04
.07
⫺.06
⫺.04
⫺.03
.06
⫺.08
⫺.02
.02
.21
.22
.17
.14
⫺.06
.03
.00
.25
.20
.24
.16
⫺.06
.03
.01
.16
.13
.13
.18
⫺.05
.03
.05
.17
.19
.16
.20
⫺.05
.03
.11
.01
.05
.02
.06
⫺.02
⫺.02
.03
.17
.16
.15
.13
.00
.04
.02
.18
.13
.17
.14
⫺.01
.05
.01
Note. Correlations greater than absolute value of .08 are significant one tailed at p ⬍ .01. A ⫽ Agreeableness;
BC ⫽ Building Confidence; C ⫽ Conscientiousness; CF ⫽ Concept Formation; CI ⫽ Continuous Improvement;
CUF ⫽ Customer Focus; CX ⫽ Conceptual Flexibility; DP ⫽ Developing People; E ⫽ Extraversion; EM ⫽
Empathy; ES ⫽ Emotional Stability; IN ⫽ Influence; IS ⫽ Information Search; O ⫽ Openness; PR ⫽
Presentation; PO ⫽ Proactivity; TW ⫽ Teamwork.
weaker at .60 than the traditional standard of .70, meaning that correlations with agreeableness
are likely to be somewhat attenuated due to measurement error.
For the Thinking cluster, there are just two significant personality correlations with dimensions.
Conscientiousness was positively correlated with Information Search, and Extraversion was negatively correlated with Concept Formation. Because Openness was not correlated with the three
Thinking behaviors, our primary hypotheses for this cluster were unsupported. Table 4 shows that
additional significant correlations were also observed between personality and dimensions. Furthermore, all Big Five personality dimensions, except Openness, were significantly correlated with the
self-source factor, and the correlations were moderate. The only other significant source factor
correlations were that Extraversion was negatively correlated with the superior source factor and
Emotional Stability was positively correlated with the peer source factor.
212
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
For the Developing cluster, all Big Five personality dimensions were correlated with all
developing behaviors except (a) Conscientiousness, which was not correlated with any developmental behavior, and (b) Openness, which was not significantly correlated with Teamwork. Because
all combinations of correlations between Empathy, Teamwork, Agreeableness, and Extraversion
were significant, our hypotheses were supported in relation to this cluster. In addition, Table 4 shows
that numerous additional interpretable correlations were also observed between personality and
dimensions. All Big Five personality dimensions, except Openness, were correlated with the
self-source factor. The only other significant correlation was a negative association between
Conscientiousness and the superior source factor.
For the Inspiring cluster, there were strong personality correlates of dimensions. Every Big Five
personality dimension, except Conscientiousness, was significantly and positively correlated with
each of the inspirational behaviors. Our hypotheses for this cluster were supported, because
Extraversion was significantly correlated with Influence, Building Confidence, and Presentation.
Table 4 shows that numerous additional significant correlations were also observed between
personality and dimensions. Moreover, all Big Five dimensions, except Conscientiousness, were
significantly correlated with the self rating-source factor. There were just two other significant
personality source factor correlations, both of which were negative. These were between the superior
source factor and Extraversion and Agreeableness.
For the Achieving cluster, all Big Five personality dimensions were significantly correlated with
Proactivity, Continuous Improvement, and Customer Focus, except Conscientiousness, which was
not correlated with any behavior. Our hypothesis was therefore not supported in relation to the
Achieving cluster. However, Table 4 shows that numerous additional significant correlations were
also observed between personality and dimensions. In addition, every Big Five factor, except
Conscientiousness, was significantly correlated with the self -source factor. The only other significant personality source factor correlation was between Agreeableness and the report-source factor.
In sum, despite the fact that there was more evidence of rater-source factors than dimensions in this data set (i.e., the mean-squared loading for source factors was larger than the
mean-squared loading for dimensions across all four clusters), personality more strongly
correlated with dimensions than source factors. The personality and dimensions correlations
reported here were considerably larger than the personality and dimension reported by Hoffman
and Woehr (2009), which had been predicted in the current study on the basis of broad
bandwidth personality dimensions, whereas the Hoffman and Woehr (2009) study used more
narrow personality dimensions. A number of unpredicted correlations also emerged, and, as
such, we do not interpret them. We discuss this issue later as a study limitation. Consistent with
Hoffman and Woehr (2009), except in isolated instances, personality did not predict the source
factors for superiors, peers, or direct reports. On the other hand, the story is different in the
current study when we consider the self source factor. In this instance, personality was
consistently and moderately correlated with the self source factor. Following from this result,
one possible explanation might be that source factors reflect characteristics of the rater rather
than the target of the rating, since in the case of the self factor, the same rater provided the
performance ratings and the personality assessment.
Discussion
It is a cause of some concern among I-O and consulting psychologists that, despite the effort
that goes into designing 360-degree feedback instruments, and despite the development actions
and administrative decisions made based on their outputs, if data from 360-degree feedback
instruments reflect the type of rater making the rating more than the dimensions the instruments
were developed assess (Lance et al., 2008). Indeed, these psychometric concerns may, in part,
be responsible for the typically modest behavioral improvements that occur over time as a result
of interventions based on 360-degree feedback (Smither, London, & Reilly, 2005). However,
the observation that source factors are stronger than effects attributable to dimensions, which
are the target of measurement attempts, is not limited to 360-degree feedback or I-O and
EVIDENCE FOR 360 DIMENSIONS
213
consulting psychology. This effect is seen more widely, both in other settings within our
disciplines (i.e., assessment and development centers) as well as multirater measurement
approaches in other areas of the social sciences (Lance et al., 2008). In I/O psychology, this has
prompted calls in both assessment center research and 360-degree feedback to do away with dimensions. Instead, some assessment center researchers have suggested that an overall exercise score be
provided. Analogously, in 360-degree feedback settings, an overall rater source score might be provided.
But this would require considerable changes in the way that 360-degree feedback systems are administered. A question also exists about exactly what such a score would represent and the extent to which
actionable development plans could be formed based on feedback structured in this manner. Yet many
psychologists would argue that attempting to surmount these challenges is preferable to providing
erroneous feedback to organizations and employees on dimensions that lack evidence to suggest that they
exist.
In this article, however, we have argued that if dimensions were found to be psychologically
meaningful based on their relations with well-chosen external criteria, we can continue using
them in 360-degree feedback, as is the current practice. In such cases, practitioners should carry
on with dimensions, despite the fact that dimensions explain less variation in observed scores
than source factors do. We investigated this issue by creating a nomological network around
dimension and source factors, using a large sample of managers that involved correlating
personality with both dimensions and source factors. Despite observing stronger internal
evidence of source factors than dimension effects, dimensions were more strongly correlated
with Big Five personality traits than were source factors. These findings add to our understanding of source factors in 360-degree feedback research in several regards. First, we show
the plausibility of dimensions with stronger external evidence, but weaker internal evidence,
than source factors. Second, we demonstrate that the use of broad personality factors and
dimensions may lead to larger personality–performance correlations. However, the increase in
magnitude of the correlations with dimensions was not mirrored by increased correlations with
source factors for the superior, peer, and direct report sources. Third, we show that the strongest
correlation between personality and source factors is between personality and the self source
factor. The self source factor is moderately correlated with all Big Five personality dimensions.
By and large, however, personality factors are uncorrelated with source factors for other rater
groups. Further research might therefore examine whether these source factors actually reflect
a characteristic of the rater group rather than the target of the rating.
Practical Implications
Our results suggest that, despite variance in ratings from 360-degree feedback instruments primarily
reflecting the group providing the ratings rather than the dimensions we wish to measure, the
dimension scores that emerge appear to have psychologically meaningful patterns of association
with personality, which supports their use in applied settings. This means that 360-degree feedback
dimensions from well-constructed instruments are valid for use in coaching and development, as is
common in organizations today. From a psychometric perspective, it also suggests that dimensions
might be used as an information source in personnel decisions, for example, for performance
evaluation and succession planning. The corollary of this position is that to use an overall score for
each rater group would very likely sacrifice valid information about specific aspects of performance
that is necessary for such human resource applications.
Because correlations between dimensions and personality were observed when the internal
validity for dimensions was so low, this study also suggests that modest improvements in dimension
measurement (i.e., increases in the variance in observed ratings due to dimensions rather than source
factors) may lead to substantial gains in criterion related validity. To increase the validity of
dimensions, practitioners should therefore ensure they implement approaches to increase valid
variance in ratings due to dimensions, and researchers should continue to examine ways to reduce
the erroneous variance in ratings due to source factors. Two approaches seem promising in
achieving this end.
214
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
The most thoroughly investigated approach to increasing rating validity is frame-of-reference
training, which imposes a common rating schema across all assessors (see Woehr, 1994). A second
new approach we recommend is that organizations introduce a formal assessment of the ability of
assessors to rate employee performance using 360-degree feedback instruments. Results of such an
assessment would clearly indicate to raters whether or not they meet minimum standards of
competence. In the context of assessment centers, results reported by Guenole et al. (in press)
suggest that such an assessment of rater competence reduces variance due to source factors and
increases variance due to dimensions. Given the importance of 360-degree feedback to organizations
and individuals, we see such an approach as desirable for all applications of 360-degree feedback,
from developmental to administrative applications.
It is important to recognize that our recommendation to reduce variance due to source
factors, and to increase variance due to dimensions using frame-of-reference training and
assessor examinations, is not to say that different rater perspectives on dimensions of interest
ought to be ignored. Each rater group’s unique perspective on targeted dimensions is captured
by that rater group’s factor loading on the dimension of interest. Perspective differences can,
and should, continue to be examined and interpreted while minimizing rater source effects.
However, consideration of perspective differences should be limited to examination and
interpretation of differences on 360 instruments’ intended dimensions rather than overall scores
for each rater group, because the targeted dimensions in 360-degree feedback have construct
validity. We wish to also underscore that, in making our recommendation to minimize variance
due to source factors and maximize variance due to dimensions, we are not taking the position
that source factors have no substantive meaning. The recent research we cited indicates they
may be substantively meaningful. Rather, we are taking the position that because source factors
are a by-product of our attempts to measure another construct (i.e., dimensions), their influence
on observed ratings is undesirable and should be minimized.
Limitations of this research should be considered, alongside its merits, in order to highlight
improvements future research can make. Our study made 12 hypotheses, of which seven were
supported. However, many more correlations between personality and dimensions were observed that were not hypothesized and therefore were not interpreted. More significant correlations that were not hypothesized emerged, in fact, than hypothesized correlations. While the
observation of numerous moderate correlations between dimensions and personality when
rater-source factors are so strong is encouraging, the fact that the patterns of correlations
between personality and dimensions were, at best, only moderately predicted is disappointing
because it hampers more conclusive statements about the validity of dimensions in the presence
of source factors. It is possible that other theoretical frameworks could explain some of the
unexpected associations, and future research should consider forming hypotheses based on
other theoretical frameworks that might have greater explanatory power than the approach used
in this research.
Another limitation is that the results of this particular analysis might not necessarily extend
to other 360-degree instruments, and certain features of our models need further examination.
For example, the agreeableness scale in the current sample had lower than ideal internal
consistency reliability, and replicating these results with other personality instruments would be
a worthwhile endeavor. Our CFA models were unable to distinguish idiosyncratic within
rater-group variation in source factors from variation due to the rater source itself, as we only
had one rater from each rater source. More research is needed to examine the impact of
including multiple raters from each rater group on the measurement of dimensions. We also
recommend replicating any models that involve modifications to improve fit, such as the
P-CTCM model presented for our Achieving cluster. Finally, to fully understand the relative
utility of dimension and source factors, researchers need to link both dimension and source
factors in broad nomological networks. Simultaneously extending the nomological network of
dimensions and source factors to their consequences as well as antecedents has the potential to
paint a clearer picture of how 360-degree feedback can be most effectively deployed in applied
settings.
EVIDENCE FOR 360 DIMENSIONS
215
References
Arthur, W., Jr., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related
validity of assessment center dimensions. Personnel Psychology, 56, 125–153. doi:10.1111/j.17446570.2003.tb00146.x
Atwater, L. E., & Yammarino, F. J. (1992). Does self-other agreement on leadership perceptions moderate the
validity of leadership and performance predictions? Personnel Psychology, 45, 141–164. doi:10.1111/j.17446570.1992.tb00848.x
Bales, R. F. (1950). A set of categories for the analysis of small group interaction. American Sociological
Review, 15, 257–263. doi:10.2307/2086790
Barrick, M., & Mount, M. (1991). The Big Five personality dimensions and job performance: A meta-analysis.
Personnel Psychology, 44, 1–26. doi:10.1111/j.1744-6570.1991.tb00688.x
Bartram, D. (2005). The great eight competencies: A criterion-centric approach to validation. Journal of Applied
Psychology, 90, 1185–1203. doi:10.1037/0021-9010.90.6.1185
Bass, B. M., & Yammarino, F. J. (1991). Congruence of self and others’ leadership ratings of naval officers for
understanding successful performance. Applied Psychology: An International Review, 40, 437– 454. doi:
10.1111/j.1464-0597.1991.tb01002.x
Borman, W. C. (1997). 360° ratings: An analysis of assumptions and a research agenda for evaluating their
validity. Human Resource Management Review, 7, 299 –315. doi:10.1016/S1053-4822(97)90010-3
Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual
performance. In N. Schmitt & W. Borman (Eds.), Personnel selection in organizations (pp. 71–98). New
York, NY: Jossey-Bass.
Brett, J. F., & Atwater, L. E. (2001). 360 degree feedback: Accuracy, reactions, and perceptions of usefulness.
Journal of Applied Psychology, 86, 930 –942. doi:10.1037/0021-9010.86.5.930
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod
matrix. Psychological Bulletin, 56, 81–105. doi:10.1037/h0046016
Church, A. H. (1997). Managerial self-awareness in high performing individuals in organizations. Journal of
Applied Psychology, 82, 281–292. doi:10.1037/0021-9010.82.2.281
Cockerill, T. (1989). Managerial competence as a determinant of organizational performance (Unpublished
doctoral dissertation). London Business School, London, England.
Conway, J. M. (1996). Analysis and design of multi-trait multi-rater performance appraisal studies. Journal of
Management, 22, 139 –162. doi:10.1177/014920639602200106
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52,
281–302. doi:10.1037/h0040957
Eid, M., Lischetzke, T., & Nussbeck, F. W. (2006). Structural equation models for multitrait-multimethod data.
In M. Eid & E. Diener (Eds.), Handbook of multimethod measurement in psychology (pp. 283–299).
Washington, DC: American Psychological Association. doi:10.1037/11383-020
Fleishman, E. A., & Harris, E. F. (1962). Patterns of leadership behavior related to employee grievances and
turnover. Personnel Psychology, 15, 43–56. doi:10.1111/j.1744-6570.1962.tb01845.x
Fleishman, E. A., Mumford, M. D., Zaccaro, S. J., & Levin, K. Y. (1991). Taxonomic efforts in the description
of leader behavior: A synthesis and functional interpretation. The Leadership Quarterly, 2, 245–287.
doi:10.1016/1048-9843(91)90016-U
Fletcher, C. (1997). Self-awareness–a neglected attribute in selection and assessment. International Journal of
Selection and Assessment, 5, 183–187. doi:10.1111/1468-2389.00058
Fletcher, C. (1999). The implications of research on gender differences in self-assessment and 360 degree
appraisal. Human Resource Management Journal, 9, 39 – 46. doi:10.1111/j.1748-8583.1999.tb00187.x
Furnham, A., & Stringfield, P. (1994). Congruence of self and subordinate ratings of supervisory practices as a
correlate of supervisor evaluation. Journal of Occupational and Organizational Psychology, 67, 57– 67.
doi:10.1111/j.2044-8325.1994.tb00549.x
Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory measuring the lower-level
facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality
Psychology in Europe (Vol. 7, pp. 7–28). Tilburg, The Netherlands: Tilburg University Press.
Guenole, N., & Chernyshenko, O. S. (2005). The suitability of Goldberg’s Big Five personality markers in New
Zealand: A dimensionality, bias, and criterion validity evaluation. New Zealand Journal of Psychology, 34,
86 –96.
Guenole, N., Cheryshenko, O., Stark, S., Cockerill, T., & Drasgow, F. (in press). We’re doing better than you
might think: A large scale demonstration of assessment centre convergent and discriminant validity. In: N.
Povah and G. Thornton III (Eds.), Assessment Centres and Global Talent Management. London: Gower.
216
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
Hoffman, B., Lance, C., E., Bynum, B., & Gentry, W. A. (2010). Rater source effects are alive and well after
all. Personnel Psychology, 63, 119 –151. doi:10.1111/j.1744-6570.2009.01164.x
Hoffman, B. J., & Woehr, D. J. (2009). Disentangling the meaning of multisource performance rating source and
dimension factors. Personnel Psychology, 62, 735–765. doi:10.1111/j.1744-6570.2009.01156.x
Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations. A
socio-analytic perspective. Journal of Applied Psychology, 88, 100 –112. doi:10.1037/0021-9010.88.1.100
Hogan, R., & Shelton, D. (1998). A socio-analytic perspective on job performance. Human Performance, 11,
129 –144.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis. Structural
Equation Modeling, 6, 1–55.
Kulas, J. T., & Finkelstein, L. M. (2007). Content and reliability of discrepancy-defined self-awareness in
multisource feedback. Organizational Research Methods, 10, 502–522. doi:10.1177/1094428107301100
Lance, C. E., Baxter, D., & Mahan, R. P. (2006). Multi-source performance measurement: A reconceptualization. In W. Bennett, C. E. Lance, & D. J. Woehr (Eds.), Performance measurement: Current perspectives and
future challenges (pp. 49 –76). Mahwah, NJ: Erlbaum.
Lance, C. E., Hoffman, B. J., Baranik, L. E., & Gentry, W. A. (2008). Rater-source factors represent important
subcomponents of the criterion construct space, not rater bias. Human Resource Management Review, 18,
223–232. doi:10.1016/j.hrmr.2008.03.002
Lance, C. E., Teachout, M. S., & Donnelly, T. M. (1992). Specification of the criterion construct space: An
application of hierarchical confirmatory factor analysis. Journal of Applied Psychology, 77, 437– 452.
doi:10.1037/0021-9010.77.4.437
Lievens, F., Chasteen, C. S., Day, E. A., & Christiansen, N. D. (2006). Large-scale investigation of the role of
trait activation theory for understanding assessment center convergent and discriminant validity. Journal of
Applied Psychology, 91, 247–258. doi:10.1037/0021-9010.91.2.247
Likert, R. (1961). New Patterns of Management. New York, NY: McGraw-Hill.
London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment,
self-evaluations and performance outcomes? Theory based applications and directions for research. Personnel Psychology, 48, 803– 839. doi:10.1111/j.1744-6570.1995.tb01782.x
Morgeson, F. P., Mumford, T. V., & Campion, M. A. (2005). Coming full circle. Using research and practice
to address 27 questions about 360-degree feedback programs. Consulting Psychology Journal: Practice and
Research, 57, 3, 196 –209. doi:10.1037/1065-9293.57.3.196
Muthén, L., & Muthén, B. (2006). Mplus user’s guide. Los Angeles, CA: Author.
Ones, D. S., & Viswesvaran, C. (1996). Bandwidth fidelity dilemma in personality measurement for personnel
selection. Journal of Organizational Behavior, 17, 609 – 626. doi:10.1002/(SICI)1099-1379(199611)17:
6⬍609::AID-JOB1828⬎3.0.CO;2-K
Paunonen, S. V., Rothstein, M. G., & Jackson, D. N. (1999). Narrow reasoning about the use of broad personality
measures for personnel selection. Journal of Organizational Behavior, 20, 389 – 405. doi:10.1002/
(SICI)1099-1379(199905)20:3⬍389::AID-JOB917⬎3.0.CO;2-G
Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings.
Journal of Applied Psychology, 85, 956 –970. doi:10.1037/0021-9010.85.6.956
Smither, J. W., London, M., & Reilly, R. R. (2005). Does performance improve following multisource feedback?
A theoretical model, meta-analysis, and a review of empirical findings. Personnel Psychology, 58, 33– 66.
doi:10.1111/j.1744-6570.2005.514_1.x
Stogdill, R. M. (1950). Leadership, membership and organization. Psychological Bulletin, 47, 1–14. doi:
10.1037/h0053857
Tett, R. P., Guterman, H. A., Bleier, A., & Murphy, P. J. (2000). Development and content validation of a
“hyperdimensional” taxonomy of managerial competence. Human Performance, 13, 205–251. doi:10.1207/
S15327043HUP1303_1
Tett, R. P., Steele, J. R., & Beauregard, R. S. (2003). Broad and narrow measures on both sides of the
personality–job performance relationship. Journal of Organizational Behavior, 24, 335–356. doi:10.1002/
job.191
Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2005). Is there a general factor in ratings of job performance?
A meta-analytic framework for disentangling substantive and error influences. Journal of Applied Psychology, 90, 108 –131. doi:10.1037/0021-9010.90.1.108
Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data.
Applied Psychological Measurement, 9, 1–26. doi:10.1177/014662168500900101
Woehr, D. J. (1994). Understanding frame-of-reference training: The impact of training on the recall of
performance information. Journal of Applied Psychology, 79, 525–534. doi:10.1037/0021-9010.79.4.525
EVIDENCE FOR 360 DIMENSIONS
217
Woehr, D. J., Sheehan, M. K., & Bennett, W., Jr. (2005). Assessing measurement equivalence across rating
sources: A multitrait-multirater approach. Journal of Applied Psychology, 90, 3, 592– 600. doi:10.1037/
0021-9010.90.3.592
Wohlers, A. J., & London, M. (1989). Ratings of managerial characteristics – evaluation difficulty, co-worker
agreement and self-awareness. Personnel Psychology, 42, 235–261. doi:10.1111/j.17446570.1989.tb00656.x
Yammarino, F. J., & Atwater, L. E. (1993). Understanding self-perception accuracy: Implications for human
resources management. Human Resource Management, 32, 231–247. doi:10.1002/hrm.3930320204
Yammarino, F. J., & Atwater, L. E. (1997). Do managers see themselves as others see them? Implications of
self-other rating agreement for human resources management. Organizational Dynamics, 25, 35– 44. doi:
10.1016/S0090-2616(97)90035-8
Appendix
Factor Loadings for Dimensions and Rater Sources Factors
IS
IS-S
IS-M
IS-P
IS-R
CF-S
CF-M
CF-P
CF-R
CX-S
CX-M
CX-P
CX-R
EM-S
EM-M
EM-P
EM-R
TW-S
TW-M
TW-P
TW-R
DP-S
DP-M
DP-P
DP-R
IN-S
IN-M
IN-P
IN-R
BC-S
BC-M
BC-P
BC-R
PR-S
PR-M
PR-P
CF
CX
EM
TW
DP
IN
PR
BC
.36
.60
.13
.30
PO
CI
CU
S
M
P
D
.74
.68
.85
.78
.39
.70
.34
.40
.83
.56
.86
.86
.20
.57
.11
.26
.80
.81
.90
.92
.56
.60
.31
.60
.57
.54
.79
.71
.56
.60
.31
.60
.47
.62
.85
.73
.72
.73
.42
.49
.52
.57
.79
.87
.56
.63
.60
.50
.48
.52
.68
.58
.78
.70
.50
.54
.54
.64
.69
.78
.70
.72
.46
(Appendix continues)
.59
.64
.76
218
GUENOLE, COCKERILL, CHAMORRO-PREMUZIC, AND SMILLIE
Appendix (continued)
IS
PR-R
PO-S
PO-M
PO-P
PO-R
CI-S
CI-M
CI-P
CI-R
CU-S
CU-M
CU-P
CU-R
Mean
CF
CX
EM
TW
DP
IN
PR
BC
PO
CI
CU
S
M
P
.58
.67
.68
.72
.52
.35
.70
.67
.83
.76
.65
.74
.53
.20
.35
.46
.29
.52
.52
.59
.57
.63
.62
D
.57
.53
.75
.68
.76
.87
.58
.67
.57
.36
.55
.64
.66
.82
.63
.63
.80
.84
.78
Note. We use S, M, P, and D to correspond to self, manager, peer, and direct report. BC ⫽ Building
Confidence; CF ⫽ Concept Formation; CI ⫽ Continuous Improvement; CUF ⫽ Customer Focus; TCX ⫽
Conceptual Flexibility; DP ⫽ Developing People; EM ⫽ Empathy; IN ⫽ Influence; IS ⫽ Information Search;
PO ⫽ Proactivity; PR ⫽ Presentation; W ⫽ Teamwork.
Received April 8, 2011
Latest revision received October 27, 2011
Accepted November 1, 2011 䡲