Test Topics
CBT Scheduling Approaches
Introduction
When an exam program is being converted from paper-and-pencil to computer-based testing, one decision that needs to be made is whether to maintain the current test administration schedule, or to change it in some fashion. Some of the options for CBT scheduling include fixed dates, testing windows, continuous testing, and event testing. The scheduling approach selected will impact various aspects of the exam program, including: the frequency of examinee access; the need for additional items; and the structure of organizational administration. Two of these approaches, fixed dates and event testing, may provide an exam program with an administration schedule that remains similar to the former paper-and-pencil schedule. The other approaches to scheduling, testing windows and continuous testing, usually provide increased numbers of administration dates. The greater frequency of administration dates typically provides examinees with increased access and scheduling flexibility, which most examinees find to be highly beneficial. However, an increased number of test administration dates also results in additional test security challenges for the exam program, which must be addressed.
Types of Scheduling Approaches
Fixed Dates
In some cases, exam programs elect to administer their CBT according to a traditional approach to test administration scheduling, by using only a limited number of fixed dates. For example, a test might be administered on two or three administration dates a year, and all examinees, perhaps nationally, test on those dates. This traditional approach provides a test security advantage over more frequent administration scheduling models. Since no examinee tests on a later date than any other examinee, there is little risk that specific information about test items can be passed along. However, the scheduling approach of fixed dates is the least flexible approach and fails to provide examinees with the advantage of increased access. Perhaps more critically, this approach to scheduling requires that a sufficient number of computers be available at the same time for all of the examinees that need to be tested. In many exam programs the number of candidates exceeds the number of computers available on a single day. For this reason, the exam program is likely to need multiple administration days, or even weeks, to allow all of the candidates to test.
Testing Windows
One scheduling approach that addresses the need for expanded test dates is the testing windows approach. In this scheduling approach tests are administered across a span of time. For example, an exam program might be available to examinees during a three week "window", four times annually. This approach to scheduling has the advantage of increasing the number of computers, and test administration slots, that are available for test administrations. In addition, examinees appreciate the advantage of greater scheduling flexibility. However, some organizational changes within the exam program are required under this approach, in order to provide necessary support. For example, staff availability to assist the examinee registration process may need to be extended over a greater portion of the year. Additional test forms may also be needed under this approach, to address security concerns. Furthermore, adjustments to certain data analysis procedures will need to be made. Typically, decisions are needed about appropriate data analysis time periods and examinee data that may be appropriately combined. Many exam programs treat each testing window as a single test administration, for item and test analysis purposes.
Continuous Testing
In another approach to scheduling, referred to as continuous testing, the number of test administrations is increased even further. In this scheduling approach the CBT may be made available so continuously that the approach is sometimes referred to as "on-demand testing". Examinees typically appreciate the increased access provided through this scheduling approach. However, this approach to scheduling clearly requires organizational changes in order to provide adequate test administration support throughout the year. Additional test forms are also likely to be needed, in order to provide greater test security across the expanded number of test dates. Furthermore, decisions about data analysis periods, and appropriate data to combine for item analysis, must be made.
Event Testing
The event testing approach is a highly targeted approach to test administration scheduling. Some paper-and-pencil exam programs have traditionally tied their test administration to a meeting or event of some sort. These events may be professional conferences, training sessions, job fairs, or other gatherings of professionals in the field. In these cases, examinees have long received the benefit of having the test available at a time and in a location of great convenience. A CBT exam program may also choose to use the event testing, or mobile testing, approach to scheduling. In this case, a large number of computers are transported to a meeting location and a temporary CBT administration site is set up. The logistics of arranging the temporary CBT site may provide some challenges to the exam program. However, event testing can be very convenient for the examinees and may also provide continuity with former organizational and data analysis methods for administering the exam program.
Summary
The decision about which test administration scheduling approach to adopt will effect numerous other aspects of the exam program, including the frequency of examinee access, the need for multiple test forms or an expanded item bank, and test security. It may also require changes in the exam program organizational structure, to provide support across increased periods of time. Furthermore, changes in item and data analysis procedures may be necessary. Ideally, the approach to scheduling will be selected for an exam program based upon identification of the approach that provides the best match to the needs of the exam program, as well as to the candidates and other stakeholders.