From the Item Bank

The Professional Testing Blog

Is it Time to Drop Option D? Considerations for Choosing the Three-Option Multiple Choice Item

Four-option multiple choice items are the most common item type used for credentialing exams.  These item types are chosen by most credentialing programs because candidates are familiar with them, they can be written at various cognitive levels, they are scored easily, and are good for statistical analysis.  However, as any person who has ever been tasked with writing four-option multiple choice items can attest, coming up with three plausible distractors can prove difficult.

And what about content that doesn’t lend itself to four-option multiple choice items?  Often times, the response options fit neatly into three categories instead of four.  For example, consider the following response options:

• increase, decrease, remain the same
• first party, second party, third party
• left, right, center

When writing items with responses that fit neatly into three categories instead of four categories, item writers will often add a nonsensical or implausible distractor.  Not only does this increase the reading burden of the test taker without adding any measurement validity, it could have a negative impact on the credibility of the exam.

One solution to this problem is dropping the last distractor, and adopting three-option multiple choice items on your credentialing exam.  Research suggests that three-option multiple choice item types perform about as well as four-option multiple choice items (i.e., they have comparable item discrimination and result in test validity similar to that of exams built with four-option multiple choice items).

Below is list of articles to review if you are considering making the switch to three-option multiple choice items:

A Comparison between Three-and Four-Option Multiple Choice Questions

Further Evidence Favoring Three-Option Items in Multiple-Choice Tests

Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research

If the content assessed in your certification examination lends itself to three-option multiple choice items, consider making this change.  Keep in mind that this could require retraining item writers, so work with your psychometrician to articulate the rationale for making this change so that it can be explained to item-writers, reviewers, and your candidate population.  Also be sure item writers understand that the quality and difficulty of the examination are in no way being compromised, and that using three-option multiple choice questions can be of benefit to both content developers and test-takers.

Categorized in: Item Writing