The effect of item ratio indicating differential item functioning on computer adaptive and multi stage tests

The aim of this study was to investigate the effect of the presence of items indicating differential item functioning (DIF) on computer adaptive test approaches (CAT and MST) performance under several conditions. Both computer adaptive tests (CAT) and multi stage tests (MST) were investigated in terms of the fact that how they were affected by DIF items and how they perform under differents test length (10-20-30-40 item) and DIF item ratio (%10, %20, %30). For this purpose, 5000 persons' ability parameters and an item pool of 600 items were generated with simulation method and some of the items on the pool were made DIF items by changing b (item difficulty) parameters by the researcher. Then, one CAT and two MST environments (1-3-3 and 1-2-4 designs) were generated on RStudio by using 'xxIRT', 'catR' and 'mstR' packages. Those generated environments were compared over same individuals and item pool. In this study, 36 different condition were investigated with 30 replication. After simulation process; RMSE, bias and correlation values were calculated and those values were used in order to evaluate test designs's performances. As a result, it was seen that CAT has the best measurement precision over all test lengths and DIF ratio conditions. When MST 1-3-3 and 1-2-4 panel designs were compared, it was found that 1-3-3 design has higher measurement precision in more conditions; however, it was concluded that the findings were not enough to say that it performed better than 1-2-4 design. In addition to that, CAT was found to be the least affected design by the increase of DIF item ratio. MST designs were affected by that increament especially in 10-item length. As the test length increased it was seen that RMSE value decreased and correlation value increased for all test designs. Therefore, increase in test length resulted in an increase in the measurement precision.