Comparison of ability estimations obtained from different conditions in multi stage adaptive testing

The purpose of this study is to examine the effectiveness of multi-stage adaptive tests of different designs under different conditions. In the study, two separate item pools were generated, with item difficulty parameters that were either normally or uniformly distributed. The study factors were module length, routing method and estimation method, and they were tested separately using data simulated to represent the two item pools. The simulation was conducted with the help of the mstR package in the R software. Five different module length designs were considered: increasing, decreasing, equal, second stage longest (first stage shortest), and second stage longest (last stage shortest). These designs were compared according to mean error and bias computed under the dynamic & static routing rule and EAP & ML final estimation methods conditions. The findings show that varying module lengths has an effect on the mean error and bias. However, it was observed that this effect differed under the two item difficulty distribution conditions studied. Whether the routing method is static or dynamic did not make a difference in the ability estimations when the item difficulties were generated using a normal distribution, but there was a slight difference when the difficulties were generated using a uniform distribution. Consistent with the literature, the results of this study also show that there is no single design that is best in all cases. Therefore, future studies are recommended to formulate alternative MST designs meeting various test requirements and objectives. MST designs are seen as user friendly and widely accepted by the test developers, test administrators and the examinees around the world. Given its advantages, the MST can be a useful alternative for the existing central exam designs in Turkey.