From the Item Bank

The Professional Testing Blog

Video – Why, When, and How to Use this Item Type

Overview

The video item type is useful across a wide range of content areas and exam programs. Video can be used to better represent crucial aspects of the profession. In addition, video can facilitate coverage of test content not easily measured with traditional written stems.

When to Use

Video clips can be used to display a variety of types of situations. In one type of video item, video scripts are prepared by subject matter experts to measure topics such as communications skills or ethics; these scripts are then used by professional actors who are filmed as they act out each scenario. In other situations, videos of “real people”, rather than actors, can be shot. This type of video is especially useful for content such as medical conditions that involve body movements not easily mimicked by an actor (e.g., a hand tremor due to Parkinson’s disease). Animations can also be filmed to display content such as scientific processes (e.g., aerodynamics) or the movement of a mechanical device in action (e.g., rotating gears). These last two types of video files may not include any audio track along with the video.

In a video item type, the video itself usually serves as an additional stimulus within the item stem, in the same way that a chart, table, or still image might. Some CBT delivery programs are also able to administer items in which video clips can be used as the response options. In items where the response options are videos, the stem might pose a traditional text-based multiple choice (MC) question for the examinee, who can then view the set of video clips to determine the correct answer. For example, a question about the best conflict resolution strategy might have video clips that demonstrate both an example of a correct communication approach along with videos that show alternative, poorer methods.

Issues to Consider

Certain aspects of the test development process are unaffected by the addition of video clips. When video is included in the stem of a traditional MC item, no new item analysis methods are needed. If the exam program’s item bank and CBT delivery software are already able to work with video files, then no systematic software changes may be needed either.  Other aspects of test development, especially item writing and test delivery, may be more substantially affected.

Depending on the type of video planned for the test, exam program staff may find it helpful to develop additional training for any item writers who will be learning learn how to write video scripts. Item writing guidelines that are specific to the type(s) of video item will also be valuable. For example, test security concerns may lead exam program staff to specify guidelines related to clothing choices (e.g., all medical personnel in scrubs of a certain color), while the goal of consistency across video clips may lead to guidelines regarding video locations (e.g., indoor office or lab settings only). (More information about item writer training and item writing guidelines for AITs is available here).

Another issue to consider when deciding to include video items on an exam is the need to provide appropriate accommodations. If the video includes an audio track, as most of them will, then some type of captioning or other text solution will be needed for candidates with hearing disabilities. Candidates with visual impairments may need some description of the scene (e.g., “A manager interviews a potential new hire in her office”) or of crucial physical elements in the video (e.g., “A biologist points to a graduated cylinder and a volumetric flask while discussing the measurement of fluids”).

Most exam programs that add video to the test include a relatively small number of video items; in many cases, the video files are also quite short. Since video files tend to be much larger than text or even audio files, when the exam as a whole includes a greater number of video clips, or longer clips, there can be concerns about the impact on total test time. An analysis before the test is administered can use the length of each video file to estimate this impact, while a more detailed statistical analysis of any impact on test time can be carried out after pretesting.

A second potential issue related to test delivery can arise when the total file size of the video clips is more substantial. In some cases, the videos can cause the overall size of the exam file to exceed a size that the CBT vendor can easily deliver over the internet. This may mean that the exam program staff must provide all the test files to the CBT vendor farther in advance of operational test dates, and it may mean that the CBT vendor charges an extra fee for the exam delivery.

Summary

Because of the extensive range of professional knowledge and skills that that can be beneficially reflected in video files, this item type has high utility for many exam content areas. In addition, video has a great deal of face validity to the public and the addition of video to the exam is a change that candidates tend to respond to very positively. For all these reasons, video is an item type well worth considering.

 This post is part of the series “Alternative Item Types.”

Categorized in: Industry News, Item Type, Item Writing