The role of the Assessment Centre RolePlayer
Naazneen Samsodien
Psychometrist
Agenda
•
Introduction
•
AC role-player guidelines
•
Prior research
•
Current practice: Who plays the role?
•
Improving AC quality
•
Role-player training
•
Standardised role-player prompts
•
Advantages and disadvantages
•
Conclusion: Suggestions to improve quality of role-players
2
Introduction
“Essentially, role-players have a different task as
assessors. Whereas assessors should be discrete
during AC exercises, role-players should involve
participants”
(Thornton & Mueller-Hanson, 2004).
Introduction
•
•
Role-players are a common component of the assessment
centre and yet little research has been undertaken into their
role in the AC process (Lievens & Klimoski, 2001).
Improving quality:
4
Ø
Who plays the role of the role-player?
Ø
Training of role-players
Ø
Role-Player prompts
AC Role-Player Guidelines
•
•
•
Create opportunities for the participant to show behaviour linked
to all the competencies being evaluated.
Ensure that they do not overplay or underplay a role.
Trained to understand their own role, the competencies being
evaluated, to recognise behaviour linked to these competencies,
to understand the character they will portray and the content of
the simulation.
•
Training should include theoretical input and practical exercise.
•
Attend pre and post AC orientation sessions and debriefings.
(Guidelines for Assessment and Development Centres in SA, 2007)
5
AC Role-Player Guidelines
•
•
Demonstrated knowledge and ability to play the role
objectively and consistently.
Non-assessor role-players also may be used if their
training results in their ability to play the role objectively
and consistently.
(International Task Force on AC Guidelines, 2009)
“Play objectively and consistently the role called for in
those interactive exercises where role-playing is required”
(International Task Force on AC Guidelines, 2009)
But what does it mean when these guidelines state that the role
needs to be played objectively and consistently?
Ø
People vary
Ø
Personalities vary
Ø
Role-players’ interpretation & perceptions
Prior Research
•
•
•
Systematic studies about the role-player in AC exercises
are nonexistent.
Therefore, role-players and role-player behaviour
constitute one of the least researched aspects of AC
technology (Lievens & Klimoski, 2001; Zedeck, 1986).
There are no studies on the actual effects that the
performance of role players have on AC quality.
Current Practice: Who plays the role?
•
•
Current AC guidelines do not prescribe who should play
the role.
Vague mention that non-assessors may be used.
Who plays the other person?
Who plays the role?
Survey of assessment centre practices in South Africa
Practices in SA
One on one talk simulations
%
Another participant
---
An observer
17
A role player
75
A professionally trained actor
in %
(N = 43)
4
Krause et. al. (in review) Assessment Centre Practices
in South Africa
Other
4
Key Findings:
Ø
In nearly all cases a role-player or an observer plays the role
of the other person if one-to-one talks are simulated.
Ø
Based on the guidelines, and current practice in SA, it seems
that any person is able to play the role provided they have
received adequate training to do so.
However:
“Although it would increase the costs involved in the AC
process, we suggest that a professionally trained actor should
play the role of the other person in one-to-one simulations
because it would increase the objectivity of the exercise”.
(Krause et. al.)
Improving AC Quality
Ø
Role-Player Brief
Ø
Role-Player Training
Ø
•
•
•
Role-player Brief:
Role-player should be well trained in the content of the
simulation, the character and the competencies being
evaluated.
The role-player brief should also be comprehensive so as to
ensure that the roles are consistently played in character.
The information shared by the role-player during the
simulation should be the same for all participants.
Schlebusch and Roodt. (2009) AC’s: Unlocking Potential for growth
Example Role-Player Brief
You will be Sam Winchester, Sales Manager: Global Training Solutions
(GTS)
Background on GTS: GTS is a consulting company with its head office in
Cape town. The company is divided into regions, with the Gauteng region
being the largest. The company specialises in providing training and
development solutions for small, medium and large organisations. The
company’s motto is: If you can imagine it, we can train it”.
Your Background: You are the sales manager for the Gauteng region. You
were appointed in June last year to this position from your previous
employer. You were previously a training consultant. You are accountable
for the total sales function in the region. As such, you have 10 sales
executives reporting to you. You report directly to the Regional Manager.
The AC participant is in the role of Regional Manager. You have an HR
Diploma, which you obtained five years ago from Damelin. You have not
attended any sales or management related training.
Your character:
You are a talkative, socially outgoing person who always see’s the “bright
side of things”. You enjoy being in the limelight, but get quite upset when
Role-Player Training
•
•
•
To play the role in a consistent fashion across participants
and across time, it is vital that role-players receive
adequate role-player training (Thornton & MuellerHanson, 2004).
In current AC practice, role-player training focuses mainly
on providing potential role-players with information about
their role.
They must learn to play the role objectively and
consistently but there is no guidelines/training on how the
role-player should be objective and consistent.
Role-Player Training
Observer Training
%
Consistency in Role-Playing
37
Assessment Practices in South Africa, Krause et. al.
•
Ø
Content
Training on the competencies being evaluated.
Ø
Character they are to portray.
Ø
Content of the simulations.
Ø
It might be necessary to understand some of the other simulations.
•
Ø
Method & Duration
Dependant on complexity of character.
Ø
Theoretical inputs where character and other aspects are mentioned in detail.
Ø
Followed by observation of a competent role-player in simulations.
Ø
Feedback should be given and simulations repeated till competence is reached.
Standardised Role-player prompts
AC exercises should “reliably elicit a large number of
dimension-related behaviors”
(International Task Force on Assessment Center Guidelines, 2009, p. 247).
BUT
According to Reilly, Henry, and Smither (1990),
assessors have sometimes too few observations on
which to base their ratings for some dimensions when
not enough behaviors are evoked.
Standardised Role-player prompts
Schollaert and Lievens suggest:
•
•
•
•
•
Including situational stimuli for evoking behaviour within AC
exercises.
Applicable to both task-based and person-based means of
eliciting behaviour.
Task-based: Situational stimuli might be planted in AC exercises
through instructions, videotaped scenes, or virtual reality
(Lievens et al., 2009).
Person-based: The inclusion of situational stimuli via roleplayers.
Role-players might serve as a practical means of structuring AC
exercises by consistently evoking job-relevant behaviour across
candidates.
Standardised Role-player prompts
Defined as:
Predetermined statements that a role-player consistently
mentions in an AC across candidates to elicit behaviours
related to specific job-related dimensions.
•
•
•
By using prompts a situational stimulus for evoking jobrelevant behaviour is created.
In role-player training, role-players are then taught to use
multiple standardised prompts per dimension in a consistent
fashion across candidates.
These prompts are framed in a script.
Prompt Examples
“How will you manage your time?”
Planning and Organising
“I do not agree with your ideas”
Tolerance for Stress
“I feel sad”
Interpersonal Sensitivity
Positives and Negatives
Ø
On the one hand role-players need to follow this script as
strictly as possible and are expected to use enough prompts per
dimension. Yet, on the other hand they also need to play their
role in a credible way.
Are role-players actually able to use prompts despite those
opposing demands?
Ø
Ø
Results from the Schollaert & Lievens (2010) study support that
role-players might be able to consistently evoke job-relevant
behaviour from candidates.
They predict that role-players will be able to structure the
exercise by the use of prompts, despite the demanding and
conflicting tasks inherent in a role-play.
Positives and Negatives
Ø
Accordingly, use of prompts by role-players might ensure that
AC exercises adhere to the AC guideline stating that “exercises
should be carefully designed to reliably elicit a large number of
dimension-related behaviour”
(International Task Force on Assessment Centre Guidelines, 2009, p. 247)
Ø
Ø
Role-player prompts are subtle in evoking job-relevant
behaviour.
The “role” of role-players in AC exercises can be extended.
Positives and Negatives
Ø
Ø
Ø
Ø
Ø
Despite following prompt-training, half of the interactions between
role-player and candidate were still not prompt-related.
Playing a role in AC exercises in a standardised way (on the basis of
role-player training) is not easy.
Role-players have to be permanently cautious of formulating their
prompts in a way that is not too strong to avoid being too
suggestive.
Prompts are meant to evoke dimension-related behaviour but they
still have to be ambiguous in order for individual differences between
candidates in reacting to them to be observed.
Thorough training to use prompts should be an essential component
of role-player training programs.
Suggestions to improve AC quality
The lack of research currently completed or underway in terms of the
impact of role-players on assessment centre quality offers scope for
exploration.
•
Based on current guidelines and practices, it is clear that role-players are
required to play their roles consistently and objectively but the guidelines
fail to specify exactly how that should be done.
•
A professionally trained actor should play the role of the other person in
one-to-one simulations because it would increase the objectivity of the
exercise. (Krause et. al.)
•
An AC’s construct validity decreases if an assessor is involved in one-toone talks (Thornton & Mueller-Hansen, 2004)
•
Suggestions to improve AC quality
•
•
•
•
Improvements to current practice should include adequate training
for role-players.
Role-player training should include understanding exercise content,
observation and feedback on their own performance when
playing the role.
Role-player brief’s should be standardised in terms of the content
that they describe.
The scope should include information on the role they are playing,
the role the candidate will be playing, the organisational context as
well as the competencies being assessed.
Suggestions to improve AC quality
•
•
•
•
•
Investigating whether role-players are able to use roleplayer prompts in AC exercises?
Generally, results from Schollaert & Lievens (2010)
support that role-players might serve as a practical
means of structuring AC exercises by consistently evoking
job relevant behaviours.
By attending prompt-training, role-players were able to
substantially increase the number of prompts used in the
exercise.
By using prompts, role-players might provide candidates
with more opportunities to display the behaviours
assessors are looking for.
No negative candidate perception when using prompts.
Conclusion
•
•
•
In current AC practice, role-players are only typically
given a specific list of things to do and to avoid.
Role-players need to be trained on HOW to perform
realistically and consistently across candidates.
Although these best practices have proven their
usefulness over the years, a key function of trained roleplayers consists of evoking dimension-related behaviour
from candidates (Thornton & Mueller-Hanson, 2004).
Future Research...
•
•
The usefulness and impact of using role-player prompts in
SA Assessment Centres.
The impact on candidate ratings due to poor role-player
briefing and performance.
Any questions?
Sources consulted:
•
•
•
•
•
•
•
•
•
Assessment Centre Study Group. (2007). Guidelines for Assessment and Development Centres in South
Africa (4th ed.).
International Task Force on Assessment Centre Guidelines. (2009). Guidelines and Ethical Considerations
for Assessment Centre Operations. International Journal of Selection and Assessment, 17, 243-254.
Krause, D.E., Rossberger, R.J., Dowdeswell, K.E., Venter, N. & Joubert, T. (In Review). Assessment Centre
Practices in South Africa. In review with the International Journal of Selection & Assessment.
Lievens, F. & Klimoski, R.J. (2001). Understanding the assessment centre process: Where are we now? In
C.L. Cooper & I.T. Robertson (eds.) International Review of Industrial and Organizational Psychology vol.
16. (pp.245-286). Chicester: John Wiley & Sons, Ltd.
Lievens, F. (2007). Assessment Centres: A tale about dimensions, exercises and dancing bears. European
Journal of Work and Organisational Psychology. 21.
Lievens, F. (2008). What does exercise-based assessment really mean? Industrial and Organizational
Psychology: Perspectives on Science and Practice, 1, 117-120.
Lievens, F., Chasteen, C. S., Day, E. A., & Christiansen, N. D. (2006). Large-scale investigation of the role
of trait activation theory for understanding assessment center convergent and discriminant validity.
Journal of Applied Psychology, 91, 247-258.
Lievens, F., & Klimoski, R. J. (2001). Understanding the assessment center process: Where are we now?
International Review of Industrial and Organizational Psychology, 16, 246-286.
Ngidi, S. (2009). Assessment centres: Unlocking potential for growth. S. Schlebusch & G. Roodt (eds.) SA
Journal of Industrial Psychology. 35(1), Art. #752
Sources consulted:
•
•
•
•
•
Reilly, R. R., Henry, S., & Smither, J. W. (1990). An examination of the effects of using behavior checklists
on the construct validity of assessment center dimensions. Personnel Psychology, 43, 71-84.
Schollaert, E. & Lievens, F. (2010). Are Role Players able to Use Prompts in Assessment Centre Exercises?
Poster presented at the 25th Annual Society for Industrial & Organisational Psychology, Atlanta, 8-10 April
2010.
Tett, R. P., & Burnett, D. D. (2003). A personality trait-based interactionist model of job performance.
Journal of Applied Psychology, 88, 500–517.
Tett, R. P., & Guterman, H. A. (2000). Situation trait relevance, trait expression, and cross-situational
consistency: Testing a principle of trait activation. Journal of Research in Personality, 34, 397-423.
Thornton, G. C. III, & Mueller-Hanson, R. A. (2004). Developing organizational simulations: A guide for
practitioners and students. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Thank you!