Dexter Davies Smith examines the psychometric approach in part 4 of our Re-thinking Recruitment series.
Welcome to part 4, the penultimate entry to this practitioners guide to recruitment from an academic perspective. In previous entries we have examined the true cost of recruitment, what to consider when evaluating selection methods (I would recommend you read this entry before continuing to ensure the following makes sense) and in our last entry we looked at traditional recruitment methods of application forms/CVs and interviews.
In this entry we will exam two psychometric approaches. The term “psychometric” comes from the idea of having standardised measures (metrics) for psychological concepts. The two we will look at are Cognitive Ability and Personality.
Cognitive Ability Assessments
Cognitive Ability Tests are designed to test a candidates General Mental Ability (GMA). GMA is effectively the term used in academic papers for ‘intelligence’. The term GMA is used as a result of research looking to answer whether there are lots of different types of intelligence. For example, it was previously unclear if aspects of intelligence like problem solving skills, verbal reasoning, and working memory were independent constructs or if they were related. The research clearly supports the concept that if someone is good at one aspect, they are likely to be good at others due to an overall general mental ability.
Face Validity: Face validity is by nature subjective, however I would say most people don’t see it as a tool with high face validity – the link between testing how well someone can predict the next shape in a series of patterns and predicting how well they will perform in job where they will never have to do that task can seem very disconnected.
Predictive Validity: Whilst the link may not intuitively lead us to believing it is a good predictor of performance, academic research has shown it is without doubt the best predictor of performance. Over 500 studies have supported the predictive power of GMA with the percentage of variance in employee performance that can be predicted, ranging between 20% and in some cases up to 70%. A meta-analysis found that cognitive ability tests predict around 35% of future job performance.
This level of predictive power is dependent on the level of complexity in the job role, the more complex the role, the stronger GMA becomes as a predictor. Even in low complexity roles, GMA will predict around 15-20%.
Adverse Impact: Of all the tools we look at, cognitive ability tests carry the most adverse impact. The relationship between score generated and how well it predicts performance isn’t affected by age, gender, race, ethnicity etc. However, the mean score on cognitive ability tests does vary between different racial groups and ethnicities.
There are several reasons proposed why this is the case and they vary from definition and inclusion of what contributes to GMA through to imbalances in societal structure. This is a complex issue, one that is certainly bigger than just its relationship to recruitment and warrants more insight than I can provide in this review. If you would like to read more on this, see the citation at the end of this entry.
Practical Review: Cognitive Ability Tests have quite a few practical features in favour of their use; they can be easily administered online, are relatively inexpensive, provide clear differentiation between candidates, are easy to interpret and don’t require line managers taking vast amounts of time out to read applications. In fact, including a Cognitive Ability Tests alongside applicants CV or application form submission, would then provide you a ranked order of applicants based on a strong predictor before deciding which applications to then spend time reading.
So cognitive ability tests provide somewhat of a dilemma, they are cheap, easy to use, consistently shown to be the best predictor of job performance. However the adverse impact shouldn’t be ignored. There really is no easy answer, however I would say they are too good to be ignored and the long term implications too serious to ignore, therefore some form controlled inclusion would benefit any recruitment system. A fantastic review by Murphy (2002) can provide some practical guidance on this. Ultimately, whilst there is no right or wrong answer, there is a correct approach and that is to be fully aware of both the benefits and negatives of the selection methods we use.
To assess personality, we first need to understand what personality is and have a reliable measure it. Personality is the different behaviours people demonstrate in the same situation – if we all behaved the same all the time, there would be no concept of personality. The accepted measure for personality is the Five Factor Model, also known as the Big Five. There are various tools that claim to measure this and they would therefore likely be a suitable tool for recruitment.
There is a big difference in the mechanics and scoring of ‘Trait’ based approaches (such as measures of the Big Five) and ‘Type’ based approaches such as Myers-Briggs Typology Indicator (MBTi). The merits of both could be the subject of another blog, however for the purpose of recruitment, Type based assessments are not suitable – as usually stated in their accompanying guidance. Unfortunately it is apparent that this guidance is often disregarded as I have seen many companies advertise that they are seeking an “ESFP” to fulfil an open position – from both an academic and practical viewpoint I would strongly advise against using such methods.
For anyone looking to use personality assessments, I would recommend they have at a minimum an understanding of the big five factors, the Wikipedia entry is a pretty good introduction: http://en.wikipedia.org/wiki/Big_Five_personality_traits.
Face Validity: There is certainly an attractive element to personality assessment that gives it a level of face validity. If personality directs behaviour and behaviour is ultimately what directs performance, assessing someone’s personality seems like a great starting point.
Predictive Validity: Despite the intuitive appeal of personality assessment, as an independent predictive measure of performance, it isn’t particularly strong.
Out of the 5 factors the make-up the Big Five, “conscientiousness” scored the highest for predicting performance but that it was only able to predict around 10% of future job performance. Neuroticism has a negative relationship with performance which is slightly weaker than conscientiousness.
Adverse Impact: Adverse impact does not appear to be an issue with personality assessment with gender and ethnic differences on dimensions of personality being very small.
Practical Review: Personality assessments can vary in price based on the tool used and as with most things, you generally get what you pay for. If you are going to invest in personality assessments, I would recommend you stick to tools based on the Big Five. Whilst technological developments means they can easily be administered online with the reports auto-generated, I would be wary of practitioners interpreting these reports without being accredited in that tool or having a BPS recognised qualification.
A strength of Personality Assessment is that it is a separate construct to Cognitive Ability. By that I mean that the 10% it can predict does not overlap with the 35% that cognitive ability predicts, so you can combine them and get a higher predictive validity as a result.
So, to conclude, we have looked at two psychometrics that can both contribute to improving recruitment processes. I would stress the word contribute as over reliance on either could have drawbacks backs but a combined predictive validity approaching 40-50% is really impressive. Using this as a basis for making an informed decision on which candidates to interview could mean you spend as much time in the interview room, but the standard of candidate is higher.
In the final entry we will look at Situational Judgment Tests & Assessment Centres which often draw on all the of the discussed selection methods.