From the Item Bank

The Professional Testing Blog

The Need for a Needs Assessment: Why Every Certification Program Should Begin with a Needs Assessment

If you ask psychometricians and test developers, “What is the first step in developing a certification program?” a majority of them will respond that the first step is to conduct a job/task analysis. From a measurement perspective, they would be right. Once an organization has determined that they want to go through the process of developing a certification program, a logical first step is to conduct a job/task analysis. However, there is a critical step that every organization should take before reaching that point, although few do take it. That first step is to conduct a needs assessment.

The purposes of a needs assessment are (a) to validate the idea that there is, in fact, a certification need that would be filled by certification; (b) to assess whether there is likely to be a demand for the certification (market research); (c) to find out whether there is likely to be opposition to the certification; and (d) to get an initial sense of the parameters of the program (what kind of assessment is needed, who needs to be involved, etc.) .
If the process confirms the need for an assessment, it will likely also provide useful information for the organization to plan its program development and marketing strategies.

Here is the overall process:

In this process, one would begin with an initial focus group, the purpose of which is to identify all of the factors that should be evaluated during the second phase of the needs assessment. Focus group participants should include all relevant external stakeholders. For example, if one were interested in developing a certification for nurses, one might include the nurses who would eventually be targeted for the certification, hospital HR staff who might make hiring decisions based on whether or not someone has a certification, physicians who may work with the credentialed nurses, patients who may choose one medical facility over another based on the percentage of credentialed staff, or insurance agencies who may offer discounts to credential holders.

During the focus group, a trained facilitator would lead the participants in a discussion about:

• the purpose of the certification program;
• the intended examinee population;
• test administration considerations (i.e., should we offer the test on computer vs paper and pencil administrations);
• reasons for why someone may want to obtain the certification (e.g., do individuals just want letters behind their name or are they looking to increase their knowledge within their profession?);
• potential demand for the certification program;
• initial certification and recertification requirements (e.g., do those who want to obtain the certification need to have formal education?);
• recertification timelines (e.g., how long is the certification valid before someone needs to recertify);
• the potential cost for certification (i.e., what is someone willing to pay for this certification?); and/or
• the scope of the certification (e.g., what should and should not be included in the certification).

Each of these factors may impact whether or not there is a need for the certification program. Once identified, a survey should be drafted. The survey questions should be close-ended whenever possible. The options for all close-ended questions should be generated from the initial focus group session. For example, when determining recertification timelines, one might ask “For which length of time should the certification be valid before someone has to recertify?” and the potential response options may be “Two years”, “Three years”, “Five years”, “Seven years”, etc.. The responses presented to candidates should be based on options discussed during the initial focus group meeting.

When tabulating findings, it is most critical to calculate the frequencies of responses. If the sample of respondents is large enough, comparisons can also be made between the frequency of responses from one demographic group and the frequency of responses from another demographic group. For example, if one is considering developing a certification program for those who design computer networks, it might be worthwhile to compare the frequency of responses for those who have other existing computer network certifications with those who do not have any existing certifications.

Lastly, the findings should be reviewed by a panel of mostly internal stakeholders (with a few external stakeholders). Internal stakeholders may include board members and staff who may be tasked with working on the certification. External stakeholders may include one or two subject matter experts from the intended audience for the certification. When reviewing the results of the needs assessment, findings should be weighed against each other. For example, there might be a high demand for the certification from the intended audience, but the intended audience is so small the cost of maintaining the certification exam outweighs any revenue generated by the certification program. Or the scope of the certification might be different than the organization originally thought, and the new scope may not fit with organizational objectives. Again, all relevant factors should be considered together to make a determination as to whether or not there is a need for a certification.

If this process is followed, each organization should have all the information it needs to make an informed decision about whether or not to develop a new certification program.

Categorized in: Industry News