To improve your test candidate experience, look no farther than direct feedback from your test candidates. Test candidates have opinions about the exams they take. Be open to these opinions because they represent direct feedback from your target market.
True, some candidates may have failed the exam and are just venting their frustration in an emotional way. But if comments or complaints cluster around the same issue(s), take the time to consider that they could be valid and, if implemented, could improve the test-candidate experience.
The Five Most Common Test Candidate Complaints
1. The test didn’t accurately measure the content required for the job/role/task. Psychometricians work with a client to ensure that the content of the exam or certification accurately captures the level of knowledge expected in the job role or by someone holding the credential.
Candidates who have actually been working in the job role for a reasonable amount of time may have legitimate complaints if test items don’t reflect the job’s responsibilities.
Sponsors should regularly review test material to ensure that it accurately captures the knowledge expected of a certificant. This is an ongoing task, not just a “one-and-done.” As job roles change, so should the material being tested.
2. The security of the exam was compromised. The legal defensibility of the exam can be substantially weakened when test candidates cheat or publish/sell exams on the Internet.
Psychometricians can help minimize this risk by educating clients on test-security best practices throughout the test-development and administration processes. For example, using multiple test forms is just one security measure that reduces cheating.
3. The test-taker environment wasn’t uniform. The test location must be uniformly comfortable in terms of temperature, lighting and sound. It should be distraction-free with adequate privacy (enough room between test candidates) to ensure a uniform testing experience. A psychometrician will identify uniform testing requirements for exams at a client’s site, a testing center or online.
4. Exams weren’t delivered the same way to all test takers. This can involve technology issues related to test delivery such as Internet outages, slow connectivity or other interruptions. If all test candidates did not receive the same test aids, this would also affect uniformity.
5. The exam contained scoring errors or other issues. For the test candidate, the following issues could have an adverse impact on an exam’s legal defensibility. They may not become known until after the candidate takes the exam.
- Wrong items on a test form
- Items with incorrect point values
- Changes to scoring after an exam has gone live
- Misusing test scores to draw conclusions the test wasn’t designed to measure
Test candidate complaints, while not always welcome, can serve as an early warning. This can be beneficial when cheating is suspected and where fast corrective action is needed to protect the integrity of the exam and prevent overexposure.
An exam is a living document with a clearly defined purpose and a natural life cycle. In cases where items need to be replaced or edited, psychometricians can assist in making those corrections in a way that won’t impact the exam’s legal defensibility.
