7776 South Pointe Parkway West
Suite 200
Phoenix, Arizona 85044
429-433 Pinner Road
North Harrow
Middlesex HA1 4HN
United Kingdom
Certification programs have proven to be an incredible tool for fueling business growth. The powerful benefits they generate impact all participants: certification program sponsors, certified professionals, and organizations hiring certified professionals.
Still, acceptance of a newly created certification in its target market is rarely automatic. Credibility is a precious commodity. Significant forethought and exacting effort are generally required to earn and keep it.
The certification development missteps we identify below have become quite familiar to Kryterion over the course of 20 years of psychometric consulting work!
We share these insights in the hope that they can help your program sidestep errors that can cost it dearly. Sometimes, those costs are financial. In other cases, the cost is reputational, and the damage done can be lasting, even before the program itself is well established.
A variety of market forces can spur discussion about the feasibility of developing a new certification program. We list and describe some of the most common reasons below.
Take heed. The marketplace you seek to serve with a new certification will decide for itself which reasons to certify are most compelling.
Too often, new certification developers don’t conduct enough market research to grasp their markets’ most pressing concerns. They compound the error by failing to determine if the market will pay for the value that the certification program intends to offer.
There is more to market research than assessing target market interests and requirements. The long-term viability of a certification program depends on developing a complete picture of the competitive landscape in which it will operate. As part of that process, new program managers must also achieve an honest understanding of their own competitive advantage(s) or lack thereof.
Will the certification program you envision address one or more of these common market interests?
Today’s innovative world economy is creating demand for new types of skilled professions daily. For similar reasons, respect for a variety of established professions has diminished.
In either case, advanced certifications that help those professions establish or expand their value may also increase the status of their respective professions.
However, in the absence of valid market data indicating widespread support, this objective may not prove a winner. The challenge is to clearly identify and deliver the high-level skills and expertise the marketplace genuinely appreciates.
Companies that want their brand to succeed must foster loyalty. How will your certification enable those firms to boost availability of the deliverables that their clients care about most? Quality, efficiency, reliability, expertise, and value.
As Forbes points out, “By ensuring employees are continuing to develop new skills, businesses not only reskill their current employee population, but they also attract new workers who are eager to learn and advance within the company.”
A good certification program will provide all the components necessary to chart an employee’s professional path(s), while quantifying their skills.
Members of your target market are ambitious. They’re driven to grow their careers, to achieve, to expand their horizons. Their desire for professional respect and recognition can be a powerful motivator.
Building a certification that the marketplace values and validates can be as significant to your program’s long-term success as it is to the success of its certificants.
Developing a certification program constitutes a significant challenge. For many organizations, including those whose due diligence was excellent, the responsibilities and requirements prove far more complex than expected.
According to the National Skills Coalition, credible certification programs provide professionals with the means to “equitably achieve their informed employment and educational goals.”
This means there must be “valid, reliable, and transparent evidence that the certification satisfies the criteria that constitute quality,” the group says.
NSC has identified 3 criteria that organizations should include as part of a credible program:
Certification programs, whether run for profit or not, face real marketplace risks. It is not uncommon for individuals who failed to earn a high-stakes certification to challenge that process. In some cases, a challenge can become a lawsuit!
The repercussions of a successful lawsuit can negatively impact both the programs financial stability and its brand.
To protect your program, consider consulting an attorney familiar with credentialing law before test development begins to avoid potential issues:
Professional testing standards, including best practices for certification development, involve a great deal more expertise than is commonly understood. The subject matter experts (SMEs) you recruit to help develop test questions are not, typically, trained credential development specialists.
That responsibility typically falls to the highly trained professionals known as psychometricians.
They’re the experts on developing credentialing exams that are fair, reliable, legally defensible, and consistent with testing standards.
What can go wrong? A lot. A legally defensible examination must be based on research that
Widely-accepted development processes and techniques have evolved to establish exam content and passing standards. Significant expertise is required to select and apply the appropriate methodologies needed to create a defensible exam.
Programs that create exams unsupported by such expertise can be forced to start over from scratch, if successfully challenged.
The process is typically rigorous. It requires careful and complete documentation.
The good news is that a well-designed, carefully implemented test development process helps enhance program credibility. Employers will quickly associate certificants who deliver superior results with the programs that certified them and draw conclusions.
There is a lot to do. Market research. Competitive analyses. Policy and procedure development. Ongoing SME recruitment and management. Job task analyses. Item writing workshops. Beta testing. Test form development. Standard setting.
Dr. Leslie Thomas, Kryterion’s Chief Strategy and Product Officer and a psychometrician with over 20 years of industry experience, states that the process required to build a professional certification exam typically requires nine to 12 months. For healthcare and finance industry-related certifications, it can take even longer! Certifications for the IT industry typically take less time, reflecting the rapid pace of product development and narrower scope of assessment.
And the cost? It varies depending on the type of exam (written vs. performance-based) and number of exams. According to Dr. Thomas, development of a single written exam can cost around $50K, while a performance-based exam can cost over $100K.
So much for the development costs. But what about the program’s maintenance costs? For more about that, see Mistake #4, below.
As Indeed.com explains, subject matter experts have an in depth understanding of a particular job. This includes knowledge, skill, and ability requirements, as well as other relevant information. SME’s expertise qualifies them to assist with various phases in the development and validation of tests and assessments.
It is also important that your SMEs are representative of the target population for your certification.
In fact, SMEs—even those with similar professional credentials— frequently have different strengths. An SME with academic and product knowledge will likely have a much different perspective than an SME with daily, on-the-job experience. Look to the latter for insights into “real world” scenarios. Since certifications validate the applied knowledge needed to perform the job, SMEs with real world insights are absolutely needed.
In other words, the important thing is to develop the right mix of SMEs to ensure a solid range of experience and expertise. Depending on the development process, your mix may also include at least one individual whose expertise is a near match for what professionals call the minimally qualified candidate (MQC). That’s a person who passes the certification exam with the lowest level of acceptable performance.
This SME serves as a reality check for more experienced SMEs. Without this individual’s insights and input, they might expand certification scope or expertise beyond that appropriately expected of a minimally qualified candidate.
To maintain the security of your examination(s), the SMEs involved in development of your certification must be willing to sign and abide by non-disclosure agreements.
Additionally, they should agree to waive any claim to intellectual property (IP) that they help develop for the certification program. They should also agree not use this IP for their own professional purposes. The IP then becomes the exclusive property of your program, as it should be.
SMEs that won’t agree to these terms are unlikely to have the best interests of your program at heart.
Certification program management sometimes forgets that its project isn’t done when a new certification exam makes its successful – and gratifying – debut. Nothing could be further from the truth. Implementation is just the first step in a certification program’s life cycle.
The need to maximize your program’s ROI makes implementation of an ongoing test maintenance plan and budget essential.
An annual exam “health check” ensures that the exam continues to perform as expected, year after year. This typically involves review of performance statistics with the goal of identifying test questions that must be replaced. Subsequent steps include identifying suitable replacements, modifying test forms, and adjusting passing scores accordingly.
Ongoing monitoring of pass rates can help determine whether test content has been exposed and may require replacement.
In many industries rapid innovation is the new normal. New skill sets develop. Existing protocols evolve. More efficient tools and processes replace previously accepted methods. Program management is responsible for ensuring that its exam content reflects current standards of professional practice and performance.
Inclusion of outdated exam content can quickly lead to loss of marketplace credibility. Plus, the inability to defend results that rely on outdated content can put program viability at serious risk.
Keeping test content fresh serves another, equally vital purpose. It enhances test security. The more popular your certification becomes, the more test content will be exposed, and the more likely it will be openly discussed and shared. Updating test content on a recurring basis protects a credential’s reputation and the underlying investment it represents.
As noted in Mistake #2 above, development of a single written exam can cost around $50K. The absence of a comprehensive and effective test security program puts that hefty investment at significant risk!
Failure to understand and address the full range of threats to exam content can severely damage a certification’s appeal, credibility, and value. The more valuable a certification appears, the more likely it is to attract unscrupulous profiteers. The risk to “high-stakes” exams is particularly acute.
A certification exam is considered high-stakes when its outcome qualifies or disqualifies the candidate for a highly-valued qualification or opportunity. Examples include a job, promotion, or licensure.
Consider the following list of common and potentially damaging test security shortcomings:
The terminology is confusing. But the distinctions between the two are real and have meaningful legal implications.
The more formal terms, assessment-based certificate and professional certification will help clarify the differences between them.
The objective of an assessment-based certificate is to build competence. It starts with a training program that prepares participants to achieve a certain level of performance. Participants’ whose subject matter mastery is subsequently confirmed by an assessment receive a “certificate.” And thus, the label: assessment-based certificate.
A professional certification validates existing competencies. It verifies that the recipient has the minimum qualifications required to competently perform a certain professional responsibility based on predetermined and standardized criteria. A formal training leading up to or preparing candidates for certification is not required.
There is another, equally important distinction between these two approaches.
Assessment-based certificates are typically considered low stakes. The goal of a certificate program is to ensure that a specific level of competence is achieved. When it isn’t, additional training and remediation are given until competence is reached.
Assessment-based certificates build or expand an existing professional role. Once a certificate is earned, there is no continuing education requirement. The certificate’s validity may or may not be limited to a specified date range.
Professional certifications, on the other hand, are frequently prerequisites for employment outcomes. These include hiring, promotions, raises, or permission to perform a role. For this reason, they are considered high-stakes exams.
Another important distinguishing feature is that their validity is typically limited to a specified date range. If not renewed, certifications will expire.
As just noted, professional certifications are typically only valid for a limited time and expire if not renewed. This means program management also has a responsibility to define – and communicate – certification renewal requirements before launching them. Setting candidate expectations for the program from the very start is an important step in establishing overall program credibility.
There is another important factor to consider. Professional certifications confirm a minimum level of professional competence. The disruption generated by the technological and scientific advances makes it essential to consistently re-evaluate and update the criteria your certification assess.
The skills and knowledge that constituted minimum competence last year or the year before may not be applicable next year! Don’t think that employers and your competitors won’t notice when your certificants lack the expected expertise.
You also have a continuing responsibility to track and verify renewal documentation submitted by certificants. This is necessary to ensure that each person using your designation has earned that privilege.
Depending on the complexity of the requirements and the volume of renewal applicants, process management can become quite complex. If your program offers multiple certifications, the management challenge can grow exponentially.
Nonetheless, efficient handling of these processes is an essential element of your brand image maintenance.
Creating a new certification program comes with a steep learning curve. Kryterion hopes that this summary will help you anticipate and proactively avoid the Top Mistakes Certification Programs Make during the development and implementation process.
Want to learn more about best practices and cost-effective certification program development procedures? Contact us. Kryterion would be happy to help you explore your credentialing options!
7776 South Pointe Parkway West
Suite 200
Phoenix, Arizona 85044
429-433 Pinner Road
North Harrow
Middlesex HA1 4HN
United Kingdom
Want more information? Fill out the form below and we’ll get back to you right away!