Performances of hybrid and traditional multi-stage tests under different conditions

In this study, the aim is to compare the measurement precision values of various hybrid multistage adaptive test (H-MST) and traditional computerized multistage adaptive test (MST) designs across different levels of ability distribution and test length variables. At test lengths of 24, 36 and 48 and when ability levels were right-skewed, left-skewed and normally distributed, the performances of hybrid designs with the same number of stages were compared both among themselves and with the MST serving as a reference. For this purpose, an artificial item pool of 1000 items was created, and the modules were pre-assembled. In the configuration of the H-MST designs, the middle or last stage of the test was item-level adaptive (computerized adaptive test-CAT), aligning with the study's purpose. The findings of the study indicate that test designs where item-level adaptation is applied at the end of the test, particularly in short test lengths, yield more precise results. As test length increased, the impact of item-level adaptation on measurement precision diminished. Additionally, hybrid designs demonstrated an advantage over MSTs when ability distributions were skewed to the right and left. The study examined two-, three-, and four-stage designs. It was concluded that three-stage H-MST designs were more sensitive to the stage where CAT was administered. Conversely, four-stage hybrid designs exhibited similar performances to tests where CAT was administered at different stages, especially in medium and long tests. Similarly, it was concluded that the effect size of the difference between the mean values of measurement precision decreased with the increase in the number of stages. To evaluate the designs more effectively, the change in measurement precision values calculated based on provisional ability levels estimated at the end of each stage was analyzed. It was found that the most substantial decrease in error values occurred in the first two provisional ability estimates, irrespective of the design applied. As expected, the stage in which CAT was applied had a positive effect on the measurement precision values. However, it was concluded that this effect diminished as test length and the number of stages increased.