Exams are an essential part of life for the student, job candidate, practitioner, professional, employee and manager. By definition, a certification includes an assessment, a test that measures the minimum requirements for a specific job role. Test candidates have opinions about the exams they take. Be open to these opinions because they represent direct feedback from your target market. Take the time to consider that they could be valid and, if implemented, could improve the test candidate experience. Let’s deep dive into the five most common test candidate complaints.
- The test didn’t accurately measure the content required for the job/role/task. Psychometricians work with a client to ensure that the content of the exam or certification accurately captures the level of knowledge expected in the job role or by someone holding the credential. Candidates who have actually been working in the job role for a reasonable amount of time may have legitimate complaints if a test’s items don’t reflect the job’s responsibilities. Clients should regularly review test material to ensure that it accurately captures the knowledge expected of a certificant. This is an ongoing task, not just a “one-and-done.” As job roles change, so should the material being tested.
- The security of the exam was compromised. The legal defensibility of the exam can be substantially weakened when test candidates cheat or publish/sell exams on the Internet. Psychometricians can help minimize this risk by educating clients on test-security best practices throughout the test-development and administration processes. For example, using multiple test forms is just one security measure that reduces cheating.
- The test-taker environment wasn’t uniform. The test location must be uniformly comfortable in terms of temperature, lighting and sound. It should be distraction-free with adequate privacy (enough room between test candidates) to ensure a uniform testing experience. A psychometrician will identify uniform testing requirements for exams at a client’s site, a testing center or online.
- Exams weren’t delivered the same way to all test takers. This can involve technology issues related to test delivery such as Internet outages, slow connectivity or other interruptions. If all test candidates did not receive the same test aids, this would also affect uniformity.
- The exam contained scoring errors or other issues. Any of the following can have an adverse impact on an exam’s legal defensibility. For the test candidate’s perspective, the issues below may not become known until after taking the exam.
- Wrong items on a test form
- Items with incorrect point values
- Changes to scoring after an exam has gone live
- Misusing test scores to draw conclusions that the test wasn’t designed to measure
Test candidate complaints, while not always welcome, can serve as an early-warning system from an engaged market. This can be beneficial when cheating is suspected and where fast corrective action is needed to protect the integrity of the exam and prevent further overexposure.
An exam is a living document with a clearly defined purpose and a natural life cycle. In cases where items need to be replaced or edited, psychometricians can assist in making those corrections in a way that won’t impact the exam’s legal defensibility.
Several blog posts ago, we discussed exam maintenance specifically with Trish Young, Director of Psychometric Services for Kryterion. Trish fielded our questions about maintenance, exam security and common mistakes that credentialing sponsors make. You can read that blog post here.
