From the Item Bank

The Professional Testing Blog

Alternative Item Types – The Big Picture

Alternative item types (AIT) is a catch-all term, and can refer to any item type that’s new or a bit different from the standard multiple choice (MC) item. AITs are sometimes called innovative item types or technology-enhanced items.

This post is the first in a planned series on AITs. In this post we’ll consider why program staff might want to add AITs to an exam program and a useful approach to designing the item types so that they will be successful. Future posts will address some popular AITs individually, along with other factors to consider when implementing new item types.

Why Use AITs
It’s always best to add AITs that offer a substantive improvement to your exam program. Each AIT has specific areas where it can offer advantages. In many cases, an AIT can give an exam program a better way to measure a particular area of the test content than standard items provide. This is easily seen in some instances, such as a test question that includes an audio clip of heart sounds (to test the auscultation skills of medical personnel). Some AITs can reduce or remove the potential for candidates to guess the correct response. For example, a mathematical multiple choice item can be converted to an item where the candidate types a numerical response and the computer automatically scores the entry.  Some AITs can help item writers measure at higher cognitive levels, too. For example, a hot spot item could present the candidate with an MRI scan and ask him or her to note the location of a specific pathology by clicking on the image.

The Need for a Process
When an exam program includes well-designed AITs they can provide good utility as they increase the fidelity to practice of the overall test.  However, there are all too many cases where an organization began developing a complex AIT that they were never able to implement. In other cases,  testing staff pretested new item types, but the item stats failed to meet quality criteria.

The best way to help ensure successful implementation of the AIT for your program is to follow a process for the design and development of the new item type. This is due to the fact that whenever a new item type is added to an exam program there are new and different things to consider. A test development-oriented process can help you identify and address these issues.

One process for developing AITs is the 6-step model (see JATT, Issue #1 on this bottom of this page – http://www.testpublishers.org/jatt-volume-10 ). This approach addresses standard test development needs and good communication across stakeholders from the beginning, within an iterative, Agile (http://www.agilenutshell.com/) philosophy.

The 6-step model (illustrated below) guides exam program staff as they take small steps with the new item type, learning and refining over several iterations before the AIT is ever administered on a live exam. In this way, potential problems are addressed early on and good decisions are made safely after the issues are understood. When the new AITs are delivered to candidates, they can be successful.

Note: Parshall, C.G., & Harmes, J.C. (2008). The design of innovative item types: Targeting constructs, selecting innovations, and refining prototypes. CLEAR Exam Review, 19(2), 18-25.

Popular AITs 

There are many variations of AITs, including a number of custom-developed items that are in use at a single exam program. This blog series is going to concentrate on some of the AITs that are frequently available as “off-the-shelf” options within CBT software. A few of these have even been used in the past on paper-and-pencil tests, but their availability within a CBT frequently means that they are easier to deliver or score than in their paper-test form. Some of the commonly available AITs that we’ll consider include the multiple response, video, and hot spot. Later posts in this series will address each of these specific AITs with an eye towards the best time to use them and the most crucial issues for implementing them.

This post is part of the series “Alternative Item Types.”

Categorized in: Industry News, Item Type, Item Writing