Just how much does it cost to create a legally defensible, high-stakes certification exam in accordance with best practices? Industry experts tell us that the price tag can total hundreds of thousands of dollars.
Exams like that constitute valuable intellectual property (IP). That’s why our experts recommend defining and implementing a carefully-crafted plan to protect your exam before you start creating it. First build your IP Fort Knox. Then deliver the exam bullion.
Inevitably, unscrupulous characters will target your exam, says Senior Psychometrician Zach Irwin, a Kryterion colleague. To outsmart them, you need a great game plan. Zach offers the following tips for developing your IP Fort Knox.
Create comprehensive policies that explicitly define who, when and how everyone and anyone with access to your exam – your IP – is allowed to transmit it, copy it, disseminate it.
Who includes candidates taking the exam, the staff that manage the exam program, the volunteers and consultants who help manage and/or develop exam materials.
Work with your legal counsel to create a non-disclosure agreement (NDA). Your NDA will define the terms of involvement – the do’s and don’ts – for the subject matter experts (SMEs) whose contributions you solicit. Remember, an NDA doesn’t protect your program until it is signed and returned. Before you permit an SME to start work, you must have their signed NDA in your possession.
- Your NDA should make clear that all program content and discussions are strictly confidential.
- It should also explicitly restrict discussion of that content outside of program sanctioned meetings.
- Establish policies that limit or preclude trainers and exam prep instructors from participating in exam content development, such as standard setting and item writing.
- Include a “release” section that indicates all of the content created for an exam belongs exclusively to the test sponsor. Contributors and subject matter experts who help develop that content agree to waive their right to use it.
As suggested earlier, your IP protection plan must account for everyone who comes in contact with your exam, especially candidates. Zach recommends working with your legal counsel to create a candidate agreement that stipulates acceptable behavior during the examination, prohibited behavior during and after an examination – such as discussing or exposing exam content, and the consequences for violating the agreement.
Require candidates to accept the terms of your candidate agreement before giving them access to the exam. The candidate signature or affirmation ensures that you have legal recourse if a breach occurs.
Parallel legal logic applies to all other program personnel. Make sure that anyone with the potential to access exam materials has signed a confidentiality agreement approved by legal counsel.
Legal agreements, Zach points out, important as they are, only provide recourse after exam content is exposed. For proactive protection, consider running background checks on development program personnel. Screen for legal violations that might call into question the good judgement of personnel with access to secure materials. To protect against candidates exposing IP, only administer exams in secure, proctored, environments.
Don’t overlook the proactive protection value of limiting access to your exam content. Enhance exam content security by ensuring that personnel and SMEs only have access to the information they actually need to do their jobs. “Ubiquitous access to all examination information,” Zach notes, “creates multiple points of failure and opportunities for breach.”
This is just the first of several articles planned in Kryterion’s new High-stakes Test Security Insights series. The next post investigates best security practices in proctor and test site management. After that, we’ll address the exam security benefits to be derived from data forensics and web crawling.
Have a question about the psychometric or security status of your test development project or credentialing initiative? Reach out to Kryterion’s Psychometrics Team here.
