Test Topics
CBT Delivery Methods
Introduction
A test that is administered on computer might be delivered through any of several methods. The most common delivery method for CBT programs is linear, or fixed CBT. This approach to test delivery is the method that is most similar to paper-and-pencil testing; a fixed set of items is administered to examinees, in a fixed order. A variation on fixed computer-based testing is the random CBT method. The random CBT method also administers a fixed set of items; however, in this approach the order of the items is randomized, to provide a modest test security benefit. A very different approach to test delivery is offered in the adaptive CBT method, also known as the computer adaptive test, or CAT method. In a CAT, items are individually selected for the examinee, based on his or her performance on earlier items in the test. While an adaptive CBT can result in a shorter test, it is more challenging to develop and can only be used by exam programs that have a large number of examinees.
Types of CBT Delivery Methods
Fixed CBT
The fixed CBT approach is probably the most commonly used delivery method, particularly for small certification and licensure exam programs. Like paper-and-pencil tests, fixed CBTs are usually based on classical test theory, which is the traditional approach to measurement. One advantage of the fixed CBT approach is that it is the simplest method for an exam program to implement. A second, related advantage is that the fixed CBT delivery method is also the easiest approach to explain to exam program stakeholders.
Random CBT
The random CBT approach to test delivery can be seen to be a variation on the fixed CBT, because the same fixed set of items may be administered in both approaches. However, a random CBT offers a slight test security advantage by randomizing the item order. One potential security benefit of the random CBT method is that the randomized order, and particularly the difference in initial items, may cause examinees to assume that they have been given different tests. This may then reduce the likelihood of an examinee passing item information on to future test-takers. Furthermore, if two examinees are testing at the same time, the items displayed on their computers at any given point are not likely to be the same items. Random CBTs, like fixed CBTs, are typically based on classical test theory and are fairly simple to implement. However, a disadvantage of this approach is that the exam program loses direct control over item ordering across the test. Thus, the random CBT may not be the delivery method of choice for an exam program in which maintaining a fixed item order is considered valuable.
Adaptive CBT
Another approach to delivering CBTs is to administer the exam as a computer adaptive test, or CAT. In an adaptive CBT each examinee is initially presented with a few items of medium difficulty. If the examinee responds to those items correctly, he or she is then given a more difficult item; if the examinee responds incorrectly, then a less difficult item is administered. This pattern of adapting the item selection to the examinee's performance is continued throughout the rest of the exam. Early research on adaptive testing suggested that it was possible to maintain equivalent reliability, while cutting the test length in half. However, operational exam programs typically experience a far less substantial reduction in test length. This is due to the fact that most operational tests have numerous constraints on the items that may be selected for a given test, such as: content rules that must be satisfied; restrictions on the administration of item "enemies"; and, limitations on the number of times an individual item can be used. An adaptive CBT has greater challenges for development and maintenance than fixed or random CBT exam programs. First, the psychometric methods required to conduct adaptive testing are more complex than the traditional classical test theory methods. This also results in a test delivery method that is more challenging to explain to candidates, the Board, and other stakeholders. Furthermore, due to these greater complexities, adaptive testing is typically more expensive to maintain. Finally, adaptive testing can only be conducted on exam programs with large numbers of examinees. The underlying psychometric methods are not accurate and stable at smaller sample sizes.
Summary
A computer-based testing program may be delivered as either a fixed CBT, a random CBT, or an adaptive CBT. Each of these methods for test delivery is used by some operational exam programs and each has certain relative advantages. The selection of a delivery method should be made carefully, as it will impact other aspects of the exam program, including item writing efforts, psychometric challenges, and test security considerations.