As testing professionals, we are often communicating statistical information about items on examination forms to stakeholders involved in the examination process. This information includes a range of topics such as item difficulty and discrimination as well as simple multiple choice item option frequencies.  While it is clear to those of us in the field of certification and licensure what these statistical properties mean, they can be less clear to those stakeholders who only encounter them during an exam committee meeting or even a board meeting.  It is our responsibility as testing professionals to demystify these numbers and provide a meaningful discussion on the relevance and importance of the data examinations yield.

It's one of the most basic questions in planning and maintaining an examination program: How many items do we need to write, review, and pretest? Write too many, and you’re expending resources on inventory that will go stale. Write too few, and you can’t assemble the requisite number of test forms to your specifications.

Getting Started
To answer the question, you need to review the rules you have established for yourself – your set of constraints.

Professional Testing is thrilled to be presenting at several sessions at the upcoming Innovations in Testing conference, March 20–23, 2016, in Orlando.

In licensing and certification tests, brevity is considered a virtue. Here’s the stem of a raw item that lacks this virtue.

The driver of a midsized sedan is pleased with the number of miles per gallon of gasoline the car consumes in highway conditions, but is unhappy with the amount of gasoline consumed in city driving. After changing the car’s oil and checking the tire pressure, the driver decides to look at the octane rating of the gasoline. Which of the following grades of gasoline is likely to provide the driver with the most economical gasoline use in city driving conditions?

A test security flowchart is a visual protocol that is used to process anomalous (or aberrant) response vectors or incidents, and to do so in a consistent, controlled, unbiased way that contributes to the due diligence of assuring the integrity of resulting scores. By creating a flowchart for each recognized threat to a given program, anomalous or aberrant results can be processed consistently and fairly.

Testing programs are built to discriminate. Licensing and certification tests, specifically, class people into two groups: those who receive the credential and those who do not. The idea is to discriminate on the basis of relevant factors (“Does the candidate now have the knowledge required to perform the task at the required level?”) and not on the basis of irrelevant factors – whatever they may be.

For the people who take your test, after all the studying and stressing out, nothing beats getting a certificate with a shiny gold seal in your mailbox. That makes a passing score report fairly easy to design. It’s going to be a variation on “Hooray! You made it!”

Chances are, though, that not every candidate is going to get that letter. What are you going to tell the candidate who fails?

Here are eight pro tips: