Ben Schwencke, chief psychologist at Test Partnership, a psychometric assessment and pre-employment testing service provider in the UK, joined the FM podcast to discuss common recruitment challenges facing employers, what psychometric testing entails and how it’s implemented, and the use of artificial intelligence in recruitment practices.
In an earlier FM episode, an expert discussed how businesses are eager to embark on AI initiatives despite remaining wary of its pitfalls.
What you’ll learn from this episode:
- Why an assessment of recruitment challenges in a business depends on who you talk to in the organisation.
- An explanation of psychometric testing and common concerns companies may have in using such testing.
- Why Schwencke considers it beneficial for job candidates to be allowed to take a practice test version of a psychometric assessment.
- The “two sides” to the use of AI in the recruitment process.
- Why recruitment methods should include tools that both incorporate AI and are resistant to it, according to Schwencke.
Play the episode below or read the edited transcript:
— To comment on this episode or to suggest an idea for another episode, contact Neil Amato at Neil.Amato@aicpa-cima.com.
Transcript
Steph Brown: Hi, listeners. Welcome to the FM podcast. I’m Steph Brown. On this episode, I’m joined by Ben Schwencke, chief psychologist at Test Partnership, a UK-based psychometric assessment and pre-employment testing service provider. We will be discussing some of the common recruitment challenges facing employers, what psychometric testing entails and how it’s implemented, and the use of AI in recruitment practices. Welcome to the podcast, Ben. Thanks for joining me.
Ben Schwencke: Thanks so much. Thank you for having me.
Brown: In your experience from working with employers, what are the main challenges they’re facing in the recruitment process?
Schwencke: It’s a good question, and it does depend on who you talk to in the organisation. When I talk to hiring managers, for example, there are very different requirements, different priorities, to someone like a talent acquisition specialist. They just need people. They just need to plug a hole in the teams affecting their performance. They’re often thinking about things far more tactically. The main thing is just the number of people that are coming through, the speed at which you can hire them, hiring velocity. The quality of a candidate as long as it adequately sufficient.
Whereas, if I’m speaking to a talent acquisition specialist, someone whose job is very much to think about this strategically, it’s much more broad. Think of quality of hire, the candidate experience. They obviously want a decent pipeline of candidates, while still keeping in mind things like time to hire. But often they’re dealing with the much more strategic issues, stuff that might impact you down the line, whether you’ve got sufficient quality of hire, whether people are trainable, and of course, ensuring that you have enough applicants in order to meet those needs.
They vary significantly, but as a general rule, people are facing more challenges now than historically they have been, and that’s more or less across the board.
Brown: Tell me about psychometric testing. What does that involve and how can it be implemented effectively?
Schwencke: Ultimately, it’s important to keep in mind that all of the predictors and the main drivers of performance in the workplace, when it comes to individual performance, they’re almost all psychological in nature. What do I mean by that? Knowledge, personality constructs, attitude, soft skills, cognitive abilities, problem-solving abilities. These are all psychological constructs, and they are the main drivers of performance, especially in technical, professional, managerial work, which is often where we’re focusing.
Consequently, psychometrics just means psychological measurement. That’s what it boils down to. It’s just measuring the things that we know are influencing performance so that you can then hire more effectively because you can easily isolate the specific characteristics, screen for them using assessments ideally because that’s the most robust way that we can do it, and then using that information to predict who is going to be an effective employee. It’s really that simple. It doesn’t have to be more complex than that. That’s how they’re used, that’s why they’re used.
And implementation varies, depending on the role, the level, the volume of applicants. But if you can sneak those assessments in, it means you’ve just cut through the noise and just measure the specific characteristics that we know based on a century of evidence now relevant to performance in the workplace. You can improve your quality of hire, speed the process up, especially now everything’s digital. It benefits everyone when done appropriately.
Brown: What are some common concerns companies have about using psychometric testing, and what is driving those concerns?
Schwencke: It does depend on how informed the client is, their opinions and experiences. We get objections all the time, as you can imagine anyone would do. One, of course, is their concerns that the assessment may have some in-built bias against people from specific demographics, whether that’s cultural, whether that’s gender, whether that’s neurodivergence. This almost always comes from a place because they don’t know.
Additionally, people have issues with things like candidate experience because obviously, you don’t want to lose candidates. If you give the assessments, and the candidates react negatively, then that’s your candidate pool negatively affected. Not ideal. Again, it’s a concern that people have when they’ve never used the assessment before because they got the baseline. They don’t know what will happen. I see this all the time, and I see very good completion rates. I see very engaged candidates, but they don’t know that.
Then the other one is, of course, relevance. Again, this very much comes from a place where they don’t know, so they assume the worst. They don’t know these assessments are effective tools. Sometimes they think they’re only for development rather than for recruitment. Almost all of the objections are just a default concerned option, and as a result, people need to take time to independently seek advice, do their research, and find providers that can speak honestly about these concerns because they’re good concerns. These are the things you should be thinking about when choosing a provider or choosing an assessment, but you shouldn’t default to the negative option. Take some time.
Some assessments absolutely will have some bias. Some assessments have a poor candidate experience. It’s a quality control issue. Generally speaking, the more informed the client, the less they have these concerns.
Brown: Going on from concerns around bias. When candidates are organised through pre-employment, psychometric testing into those roles, how can the risk of bias be minimised in the later stages or interview stage?
Schwencke: The interview stage is a really tricky one. One of the good things about psychometrics compared to, say, interviews is that you can do research because it’s a standardised product and assessment that you can give to thousands of people. It’s very easy to do research to find out if there are differences based on characteristics, if there are certain questions which seem to be behaving differently. Harder to do that with an interview because it’s so informal.
That’s a really good question, and it’s really tricky to do. We know, particularly when it comes to neurodiversity, interviews tend to be a real killer, particularly for candidates on the autistic spectrum. What we’ve seen in the research, disclosure does help. That’s important to know because it could easily have gone the other way, and disclosing a diagnosis might make you more likely to experience discrimination.
From the research, it seems to be the opposite. If you’re a divergent, it is, in my opinion, worth sharing with the interviewer, then they will understand, and they will, generally speaking, try to incorporate that. Whereas pretty directly if you keep that quiet and then you break all the sort of tacit rules: don’t make eye contact, you face slightly away. That could be misinterpreted.
It can be really tricky because it is all down to just minimising the bias of the individual person. Then people vary so much. It’s not like you can click a button and get extra time or change the contrast of the screen or text size, like you might do. There’s an adjustment for dyslexia on an assessment. It’s harder to do.
It’s more about informing people, training people as well, because very few people receive proper training on interviews. The more structured the interview, the better. If it’s just purely conversational, then that just allows bias. Training people, disclosing, particularly neurodiversion, anything that might impact how conversation might go, that is about as good as you reasonably can get at this stage. Ideally, in the assessment stage, we’ve done a lot of heavy lifting beforehand, and as a result, at the end, it should ideally be just quality control, a tick box-type exercise where most people will make it through.
If you’re not prepared to make an offer to most of the people you interview, you probably haven’t screened heavily enough. In doing that, you make more affordances and require less room for bias.
Brown: There’s a lot of recruitment challenges, of course, facing employers and companies, considering that a lot of job seekers will be impacted by those processes as well. How can companies encourage job seekers who have had little opportunities in competitive markets to engage with pre-employment assessments like psychometric testing?
Schwencke: One thing that I really recommend, something we do a lot, is encouraging practice and preparation. Now, even in psychology spaces, this is a little contentious because people talk about things like practice effects as if it was a negative thing. That if you just give people the opportunity to prepare, you artificially inflate the score, and it makes it less reliable. That’s the opinion. A lot of organisations follow suit and think, well, we want to know their true level of ability. We don’t want to know just how well they can prepare.
I’m of the opinion that’s not how it works. I think what happens is when you thoroughly prepare people, they get less anxious. They get less stressed. They reveal their real levels of ability more precisely when you give them the opportunity to practice because the difference between pre- and post-practice scores, it’s all just a non-assessment variance. It’s all just how stressed they were, silly mistakes. They misread the instructions the first time. It’s going to be more reflective of their actual ability.
This is particularly true with cognitive assessments: verbal, numerical, logical reasoning assessments, cognitive ability assessments because they really underpin one’s ability to learn. If you give people the opportunity to learn and to practice and develop on something as specific as doing these tests, what happens is, of course, the people who are more cognitively able will improve their scores even more.
That’s the whole point because they benefit from preparation, which is important, and it’s one of the reasons why we use the test in the first place. If people are concerned about attrition, if they’re concerned about some people being inherently disadvantaged, there is a big socioeconomic component to this because the people who are given the most preparation materials, the most support, it’s often people from more privileged backgrounds. You think the career support you would get at Oxford compared to a local non-target university is going to be quite different.
So, preparing people and giving them as much resources as possible, or making it as equal as possible, everyone that applies gets the same access to resources, then two things happen. One is the scores are more reflective of their actual level of ability, and of course, the people most invested in the process try the hardest. I think it’s very much down to the employers to make people feel welcome and supported and give them what they need to succeed.
Brown: The use of AI in recruitment has mixed views. What are your thoughts on how that is currently being used?
Schwencke: There’s two sides to this one. There’s, of course, how organisations are using AI, and then there’s, of course, how candidates are using AI as well. With organisations, it’s a really tricky one. They’re still really finding their feet. I attend so many conferences and talks, and people talk about this in such an abstract sense, with no actual, clear guidance on how. Only, this is going to be revolutionary. This is going to be really exciting. It’s going to change how you do your work. Never, specifically, how it’s going to do that. At the moment, large language models can be really helpful when drafting things, proofreading, helpful pair of eyes.
When things get more strategic, it gets a little concerning, in my opinion, because people are often defaulting to things like an AI screen for CVs and résumés, which certainly in the UK and EU is a very risky thing to do, GDPR-wise. Automated decision-making, especially something as black box as AI, that you cannot justify. You have no idea what the logical process was or why someone put it through.
A really interesting study came out recently, looking at this CV screening using ChatGPT, and it found a very consistent bias that the first CV upload is more likely to be selected. It doesn’t matter which one it is, as long as it is the first, more likely to be selected. It’s a real first-come, first-served bias. It’s indefensible.
If someone challenges you and takes it to a tribunal, how are you going to defend that when even the organisation has no idea what the decision criteria would be? It’s very risky, and we’re in really uncharted territory. There isn’t a lot of guidance.
Then when it comes to candidates, it’s even more concerning, in my opinion, because particularly when it comes to assessments, particularly it comes to video interviews, and especially CVs and application forms, it’s nullifying a lot of those selection criteria because previously, you would read your CV, and if it’s well written, it has all the buzzwords or whatever, they make it through.
But now, they’re all written by ChatGPT, and it’s a real positive feedback loop because it allows you to make more applications. We’re seeing a deluge of applicants with similar-looking but well-written CVs. It’s almost becoming a bit of an arms race here, which nobody’s going to win.
Brown: Thinking about how recruitment processes could evolve in the future, do you see any pre-employment assessments ever working with tools such as AI, and how that might advance those technologies and those tools in the future?
Schwencke: There’s two things here. One is some sort of assessment that incorporates AI, and the other is finding tools which are resistant to it. The first is a little bit of a holy grail, I would say, because we’re increasingly using AI anyway. I use it all the time. I’m not anti-AI.
And it’s a skill issue. Doing good prompts, working with it in order to achieve goals. The problem is that it’s so varied, trying to build an assessment, which is test your ability to use large language models when there’s so many of them. And the rate of progress is so rapid, people will be using different models even within ChatGPT, let alone across Gemini, and it’s very difficult to keep consistent. It’s very hard to build an assessment that uses AI as part of its core functions. But I’m holding out hope someone figures it out. More realistic is going to be assessments that are more resistant to AI.
One of the things that we do, and that other providers do, is we have assessments which are fundamentally gamified. They use more dynamic elements, and particularly the cognitive assessments, where this works pretty well. Things moving across the screen. It uses game mechanics, which nullify your ability to take a print screen and then upload it and then ask ChatGPT to answer. You’ve missed three questions by the time you’ve done that, and it didn’t incorporate the information anyway. That’s a real moat at the moment, and so people should very strongly consider that as an option.
That being said, most people are honest, and we haven’t seen a massive uptick in scores, for example, globally across our assessments. It doesn’t look like people are cheating significantly with AI, but it is a vulnerability because you can negate entirely using gamified assessments.
I see things moving in that direction until we figure out how you can incorporate it constructively. I see people doing more and more on the technical front, the psychometric front, finding ways to minimise the benefit of trying to cheat with AI and building assessments and selection processes to optimise for that.
Brown: We’ve talked a bit about how those tests can be used in the recruitment stages. Can those models be used to address things like skills gap challenges to upskill people depending on their unique abilities and differences?
Schwencke: They certainly can. The late stage of the process is very much the time to do that, either just before the offer is made or even post, part of the onboarding process. The reason I say that is when it comes to skills, hard skills, knowledge, that sort of thing, incorporating that in the recruitment process can be tricky because there’s a time horizon component.
If you imagine a big grad scheme, for example, or apprentices or interns, what they know now is obviously not going to be reflective of what they will know in a week, a month, a few months, because they’ve been trained.
The old adage was always hire for attitude, train for skills. That does hold true, but at the same time, particularly for experienced hires, you do want them to hit the ground running a little bit. That’s particularly true if it’s a contract, if it’s interim.
What they do know isn’t always offset with the potential. You have to balance both sometimes. That’s why you’re doing it at a late stage, technical interviews, or more traditional hard skills testing. It makes sense towards the end, once you know the potential is there. It’s a real cherry-on-top stuff, but that information does, of course, reveal skills gaps, highlights where L&D spend should be used.
This can be done strategically, too, in that you have a cohort coming in and you say, OK, it looks like they’re really struggling with Excel, whatever the skills gap may be, there’s forecasting that can be used for that. There’s a lot of opportunity there, particularly in that front. Indeed, behaviourally as well, if you’re using behavioural assessments, you can see, OK, it looks like this cohort has less resilience overall compared to previous years. We now know where we need to be focusing. They have less creativity, whatever the characteristic may be. Whatever the gap may be, identifying it both at the individual level and at the cohort level has a lot of benefits, and it makes sense to use that information.
Brown: Thanks so much. Is there anything that we haven’t touched on in this conversation that you think is important to add into this?
Schwencke: Everything I’ve said, it does hinge on the provider doing their job, which is not always the case. Psychometrics are very effective tools when they are developed effectively. The product’s like any other, and to some degree, you do get what you pay for. There are a lot of providers out there who have completely skipped all the necessary R&D, haven’t done any of the stages that would otherwise be required to save money, or because they don’t have the expertise. You can develop assessments that are extremely fair, extremely effective, really good candidate experience, really robust against AI usage. Or you can develop assessments which do none of those things.
When I talk about psychometrics, I’m talking about the idealised version of them. There are plenty of tools out there that do not meet those characteristics. So, buyer beware. Take some time to really vet the provider, the evidence that they say is true, and they should be able to provide it. Should be able to provide psychometric research, technical manuals, R&D, to prove that the product meets the standards of which it is claimed. I’m certainly not an apologist for people who are not doing their job. There are plenty of tools out there that do not work, but the method is sound, you just have to do your due diligence.
Brown: Thanks so much, Ben.
Schwencke: Thank you very much for having me.
