Co-authored by: Reed Castle, Ph.D. and Vincent Lima

Certification examination programs often have more than one examination form (set of questions, or items) in use at a time, and they require routine updates. The exam forms represent the same content, but have different items. As new items come into use, they may be easier or harder than the first set of items. As a result, exam forms can have varying degrees of difficulty. These variations need to be taken into account when reporting scores and pass/fail decisions over time and exam forms.

Occupational Licensing’s Rise to Attention

In the United States, the post-1950 expansion of the service economy fueled policies by States to license specialized skill-intensive occupations in rationally uncontested areas such as medicine and law. Subsequent progressive social reforms such as the consumer activist movement of the 60's contributed to a public panglossian view of occupational licensure as a benevolent action enacted to protect them against unscrupulous, incompetent and unsafe providers of professional services. Legislators found little political reason to constrain proposals for licensure particularly when buoyed by the fact that licensing is principally self-funded through licensing fees giving it little if any impediment from budget or appropriation constraints.

While conducting research for a journal article, I came across this definition of “certification”:

“Certification refers to the confirmation of certain characteristics of an object, person or organization.  This confirmation is often, but not always, provided by some form of external review, education, assessment or audit.  Accreditation is a specific organization’s process of certification” (Wikipedia). 

This is a continuation of the previous blog article that discussed the use of performance examinations for certification and licensure.  To summarize the previous blog, performance examinations (i.e., “hands-on”) are expensive to use and maintain. Many certification and licensure authorities have abandoned hands-on performance examinations for some alternative measure of a person’s skills. However, there are situations where the expense and effort are worth the expenditure because of consequences to clients.  Situations where a performance examination may be worth the expense are those where failing to measure an examinee’s performance skills (fine motor) could result in pain or suffering for other people (e.g., patients). 

The video item type is useful across a wide range of content areas and exam programs. Video can be used to better represent crucial aspects of the profession. In addition, video can facilitate coverage of test content not easily measured with traditional written stems.

With the release of the micro-credential white paper “Micro-Credentials: Preparing a Skilled Workforce in Real Time,” which you can download here, we are bringing you a series of excerpts from the paper that we think are important to highlight.

Reed Castle, Ph.D.

The primary goal of any testing program is to treat test takers in a fair and equitable manner and to assure that inferences made from an exam score are appropriate. To do so, one component is to maintain proper scoring and reporting procedures. 

If you ask psychometricians and test developers, “What is the first step in developing a certification program?” a majority of them will respond that the first step is to conduct a job/task analysis. From a measurement perspective, they would be right. Once an organization has determined that they want to go through the process of developing a certification program, a logical first step is to conduct a job/task analysis. However, there is a critical step that every organization should take before reaching that point, although few do take it. That first step is to conduct a needs assessment.

Examinations can be split into two types: norm referenced and criterion referenced.  Norm referenced examinations compare each examinee’s score to a normative sample.  Criterion referenced exams compare each examinee’s score to an established standard.  Let’s think of each of these types of exams in terms of running a race.  Typically, when people come out and participate in a race they are being evaluated based on the performance of everyone else in the race. This process is similar to how examinees are evaluated on a norm referenced examination.  Whoever crosses the finishes line first will be considered the winner of the race regardless of what time they cross the finish line.