From the Item Bank

The Professional Testing Blog

Certification and Licensure Examinations: Norm Referenced or Criterion Referenced?

Examinations can be split into two types: norm referenced and criterion referenced.  Norm referenced examinations compare each examinee’s score to a normative sample.  Criterion referenced exams compare each examinee’s score to an established standard.  Let’s think of each of these types of exams in terms of running a race.  Typically, when people come out and participate in a race they are being evaluated based on the performance of everyone else in the race. This process is similar to how examinees are evaluated on a norm referenced examination.  Whoever crosses the finishes line first will be considered the winner of the race regardless of what time they cross the finish line.  They are only being compared to those who participated in the race, not to a set standard.  Now let’s think about running that same race in terms of criterion referenced exams.  If races were evaluated based on criterion referenced exam standards all race participants would be evaluated based on a pre-established standard.  For example, when using criterions referenced exam standards it would have been decided ahead of time that anyone who crosses the finish line in 30 minutes will “win the race” or win a medal.  Therefore, the speed of the participants in the race does not matter in comparison to each other and only matters in terms of meeting or exceeding the established standard.

Certification and licensure examinees’ scores are being evaluated against an established standard, therefore, these examinations are considered criterion referenced.   Specifically, certification and licensure examination programs are evaluating examinees minimal competence in a specific field.  There are several advantages to administering criterion referenced examinations.  One, the passing standard doesn’t vary based on the ability level of the testing group. Criterion referenced examinations set standards based on a level of knowledge that examines need to have in order for the larger population to feel comfortable with the examinee holding that credential or license. Therefore, the ability level of the testing group should not impact whether or not the examinees obtain the credential or license.  Two, the passing standard does not vary from administration to administration.  When the passing standard is based on a set of amount of knowledge needed it will not matter when that standard is assessed.  For example, if a candidate tests in the Spring 2015 window or they test in the Fall, 2015 window they will still be assessed against the same passing standard.  Similar to the standard not varying based on ability level of the testing group, it should not matter when an examinee tests or for that matter on which examination form.  Standards are established to distinguish between those who are minimally competent and those who are not, therefore, when and what form an examinee tests on should not impact whether or not they obtain a credential.  Finally, when using a criterion referenced standard the meaning of the certification remains constant. Those who have obtained the credential or license in the past can feel confident that those who obtain the credential in the future are being evaluated against the same standard, and are being asked to demonstrate the same level of competency.

In order for the advantages of criterion referenced standards to hold true, and for certification and licensure examinations to remain credible, it is critical that organizations understand the importance of setting the passing standard at the right point.  If the standard is set too low individuals who are not qualified will pass the exam, and if the standard is set to high individuals who are qualified will not pass the exam.  Certification and licensure organizations need to be aware of how these risks can affect their organizations and credentials.  As such, credentialing and licensure organizations should evaluate the potential risk of passing a candidate who is not minimally qualified, as well as the potential risk of not passing a candidate who is minimally qualified.  For example, if an individual will be operating in a profession where there is limited supervision and a high risk to public safety, such as a doctor, then the need to define minimal competence is highly important due to the public safety risk involved in credentialing someone who is not minimally competent.   Failure to do this correctly can result in certification and licensure organizations losing the trust of the population they serve.

Categorized in: Industry News