From the Item Bank

The Professional Testing Blog

Innovative Item Types: Strengths and Weakness

Adrienne Cadle, Ph.D. and Cynthia G. Parshall, Ph.D.

Do you have multiple choice items that are not accurately assessing the content of an exam?  If so you may want to implement innovative items.  In our workshop at the 2015 ATP Conference we presented four innovative item types, along with when each one should be used.

The key factor that should be considered before adding an innovative item type is whether it can improve the measurement quality of the exam.  The first step in implementing innovative items is choosing the right type.  To help guide your decision on which types to implement, it is helpful to have a general understanding of the structure, strengths, and weaknesses of each item type.

Short Answer Items
Short Answer items have a stem that asks the test taker a question and the test taker responds to, or answers, the question by writing or typing a response.  This is a constructed-response item type because test takers are constructing their answer.  Some of the strengths of this item type are that they are efficient in measuring certain content, they measure recall rather than recognition (as with multiple choice items), and cueing and guessing are largely removed (since the candidate is constructing their response instead of choosing from a list of responses).  One weakness of this item type is that it can be difficult for item writers to write stems that are clear, yet avoid cluing; another is that short answer items can be difficult to score automatically (i.e., on a computer-based test) due to the potential variations in responses.

Audio Items
An Audio item is an item in which test takers listen to audible content, then respond to a question or prompt usually by selecting the correct response from several response options.  Some examples of audio sounds include speech sounds, music, physiological sounds, and equipment or machinery.  One of the most significant strengths of the Audio item is that it can expand test content into new regions.  For example, an item on a medical assessment could include lung or breath sounds to measure auscultation skills, rather than providing a text description of the breath sounds.  One consistent challenge when adding Audio items is the need to provide alternatives for examinees with hearing disabilities.

Hotspot Items
A Hotspot item is basically a graphical multiple choice item.  Test takers answer the question asked of them, or respond to the prompt, by clicking somewhere on an image.  In most Hotspot items there is only one right choice.  Some of the advantages of this item type are that they are very good for measuring visual content and they may provide better fidelity by allowing the test taker to respond directly on the item (e.g., asking an acupuncturist to click on the location on which he or she would place a needle).  Some of the weaknesses of this item type are that they often target only lower cognitive levels (i.e., recall) and they may be difficult to interpret if the images used are poorly sized or ambiguous.

Drag and Drop Items
Drag and Drop items are items in which test takers respond to a question by dragging tokens onto a region.  Drag and Drop items are often used for sequencing or ranking/prioritizing.  A strength of Drag and Drop items is that they can often be used to measure higher cognitive levels than traditional multiple choice items (i.e., analysis rather than basic knowledge or recall).  And for some content areas they can provide a more direct measurement than traditional multiple choice items (e.g., asking a sonographer to label anatomy on an ultrasound image by placing labels directly on the image).  One potential weakness of Drag and Drop items are that they can require too much knowledge and/or time for a single score point.  Similarly, the interface can be complex for test takers (i.e., it may be difficult for test takers to understand how to respond to the item).

Summary
In summary, every item type has its own strengths and weaknesses.  Whether or not an item type is appropriate for any certification exam depends on the content being assessed on the certification exam.  This purpose of this post is to help you choose which innovative item type may improve your certification exam. For more on this topic, see the series “Alternative Item Types.”

Categorized in: Industry News