WHITE PAPER
Why Personality Tests Cannot Predict
Employee Performance
The Development of the Performance Profile Questionnaire
as an Evidence-Based Recruitment Tool
Talent Chaser™: Evidence-Based Recruitment & Retention
Contents
Introduction ......................................................................................................... 1
Personality Test Construction............................................................................. 2
Behavioral Questionnaire ............................................................................... 2
Subject Group ................................................................................................. 2
Control Group.................................................................................................. 3
Development Process..................................................................................... 3
Performance Profile Questionnaire Construction (PPQ) ................................... 5
PPQ – Version I .............................................................................................. 5
PPQ – Version II ............................................................................................. 7
PPQ – Version III ............................................................................................ 7
Employee Retention...................................................................................... 10
Introduction
The world’s first personality test was developed by Raymond Cattell at the University of Illinois
in 1949. At that time, Professor Cattell indicated that such tests were not designed to be used
for screening job applicants.
Page
Since the advent of the personality test, many users have discovered that while such
tests can provide a relatively accurate picture of an applicant’s personality, it has not
proved possible to develop a meaningful link between this and the subsequent
performance of the individual.
1
In spite of this, over the ensuing years, the use of personality tests in recruitment situations has
become commonplace. It is the purpose of this paper both to explain exactly why personality
tests should not be used to screen job applicants and to describe the development of
Performance Profiling, a method specially developed to overcome the limitations of Personality
tests.
WHITE PAPER
Research into the way in which personality tests are developed has revealed that the problem is
rooted in the way in which such tests are constructed. The Performance Profile Questionnaire
(PPQ) was developed using an entirely new methodology which is described in this paper. It
overcomes the limitations of personality tests and has been independently verified for use as a
recruitment screening tool able to provide an accurate prediction of a job applicant’s role-related
performance potential.
Personality Test Construction
Tests of all kinds are developed and published by test publishers. In the US, the quality of this is
monitored via the Association of Test Publishers. This association maintains a list of accredited
test publishers. The Performance Profile developer, The Cambridge Don, is a long standing and
fully accredited member of this prestigious association.
The development of a traditional personality test involves the use of three main elements, these
are:
 Behavioral Questionnaire
 Subject Group
 Control Group
Each of these three elements is described below together with the role they each play in the
development of a traditional personality test.
Behavioral Questionnaire
A behavioral questionnaire contains questions, the answers to which yield insights into the way
in which the individual answering the questions will behave in a variety of situations. Such
questions typically link feelings to actions. Two examples of this type of question are described
below:
1. Do you feel uncomfortable if you suddenly become the center of attention?
[Yes] [Uncertain] [No]
2. Does untidiness in others make you angry?
[Yes] [Uncertain] [No]
During personality test development, members of a Subject Group are invited to complete the
behavioral questionnaire under development. These subjects are typically selected to be
Page
Subject Group
2
It turns out that individuals who answer Yes to Question (1) are somewhat more likely to avoid
social interaction and individuals who answer Yes to Question (2) are somewhat more likely to
be tidy themselves. Behavioral questionnaires used by test publishers typically deploy a range
of such questions designed to cover a diverse range of situations. To achieve a broad
description of an individual’s personality, test publishers typically deploy between 100 and 200
questions often in the form of a multiple-choice questionnaire.
WHITE PAPER
representative of the types of people who will be required to answer the questionnaire once it
has been published. Test publishers typically utilize subject groups with approximately 4,000 to
6,000 individuals.
Control Group
In the development of a personality test, the control group typically consists of friends and
associates of the individuals in the control group. These people are called upon to answer
surveys designed to elicit their views as to the personality of individuals they know in the subject
group. Typical questions used in such surveys are:
1. Which of the following best describes the amount of social interaction that
he/she likes to have with others?
[Very Little] [Little] [Uncertain] [Quite a Lot] [A Great deal]
2. Which of the following best describes him/her?
[Very Untidy] [Untidy] [Uncertain] [Tidy] [Very Tidy]
Development Process
Page
3
The actual test development process is typically broken down into three phases. These are
described in the three charts below:
Page
Personality tests are good predictors of personality because individuals of the control
group, being friends and associates of those in the subject group, are well positioned to
describe their personality. There is however a significant difference between being able
to predict personality and being able to predict role-related performance potential.
4
WHITE PAPER
WHITE PAPER
Performance Profile Questionnaire Construction (PPQ)
The development of a test that can meaningfully be used to screen job applicants requires that
the individuals in the subject group used by personality test developers be replaced by
employees working in varying roles in organizations willing to participate in the test development
process. Additionally, the control group of friends and associates needs to be replaced by
individuals who are managers of the individuals in the subject group.
The development of the PPQ involved the use of three main elements, these were:
 Behavioral Questionnaire
 Subject Group of Employees
 Control Group of Subject Managers
PPQ – Version I
At the outset, no one had attempted to use managers’ performance ratings of their subordinates
in the construction of a test that would predict role-related performance potential. As a result, it
was not certain that it would prove feasible to do this. At the heart of this was the doubt as to
whether or not sufficiently strong correlations could be identified between the answers
individuals of the subject group gave to behavioral questions and the performance ratings those
individuals obtained from their managers.
Initial research was undertaken to identify any behavioral questions that could usefully be used
in the development of the test. Questions from a wide variety of instruments then in existence
including Personality, Motivational, and Attitude and Interest inventories were reviewed. New
questions were then created covering similar areas. Version I of the questionnaire deployed
some 900 questions and took approximately 8 hours to complete.
Page
5
This questionnaire was then deployed on a research basis in a wide variety of major European
organizations using the revised methodology described in the following three charts:
Page
6
WHITE PAPER
WHITE PAPER
PPQ – Version II
During the initial research, it was established that a significant proportion of the questions in the
original PPQ could be removed without harming the accuracy of the test. As a result, the PPQ
was reduced to a questionnaire containing 475 questions. Version II of the PPQ took 3 hours to
complete.
Throughout this process, no correlations were identified, between employee’s answers to the
PPQ and their performance ratings that were sufficiently strong to allow the PPQ to be used to
predict the role-related performance potential of job applicants.
PPQ – Version III
During this part of the development, neural networks were used to improve the way in which
data was being analyzed. At that time, it was already known that the existing method known as
Multiple Linear Regression suffered from two theoretical assumptions known to be incorrect:
Page
7
 Trends between data points were linear
 Factors being analyzed were independent of each other
WHITE PAPER
Since neural networks suffer from neither of these limitations, it was thought that potentially they
could be used as the basis of a new analytical platform that would possibly enable the PPQ to
be turned into an instrument capable of predicting role-related performance potential.
To achieve this, all of the data collected at that point in time was collated into Training Pairs
made up of two groups of data:
Inputs
These comprised Answers to the PPQ and data collected through the use of a job analysis
system designed to provide information regarding the challenges faced by individuals of the
subject group in their respective jobs.
Outputs
These comprised performance ratings given to individuals of the subject group by their
respective managers in the control group.
Page
8
During this program, both the design of the neural network was varied and trained using the
revised methodology described in the two charts below:
WHITE PAPER
As a result of this research, the PPQ was reduced to a multiple-choice questionnaire containing
148 questions with the capability of predicting role-related performance. This capability was
however found to be limited to situations similar to those organizations that participated in the
research.
Page
9
To overcome this limitation, Talent Chaser was developed. This system uses the methodology
described in the chart below in which repeated performance ratings are collected and used in
the ongoing neural network analysis of organizational-specific data. Since incorporating the
PPQ into the Talent Chaser platform, it has proved possible to use the PPQ to screen job
applicants and predict role-related performance potential.
WHITE PAPER
Employee Retention
Page
“The average days in job increased from 384 in 2006 to 770 in 2012. The
increase/difference between the days in job for 2006 (average=384) and 2012 (770)
was statistically significant (F=27.02) and could only have occurred by chance less
than 1 time in 10000 (p<.0001).” Dr. Richard Feinberg, Professor of Consumer
Science – Purdue University.
10
Unlike performance ratings, which are subjective, days-in-job data in organizations where Talent
Chaser is deployed is objective and has been independently analyzed by Dr. Richard Feinberg,
Professor of Consumer Science at Purdue University. Below, is a graph covering the six-year
ongoing deployment of Talent Chaser and the PPQ within a leading U.S. organization.