Investigation of scale conversion errors in testlet-based tests

This study aims to examine the effects of testlets on scale conversion. For this purpose unidimensional item response theory, bifactor and testlet response theory models were applied to the testlet-based tests for the estimation of item and ability parameters. In order to equate the tests, the parameters were placed on the common scale using mean-mean, mean-sigma and Stocking-Lord and Haebara scale coversion methods under the common-item non-equivalent groups design. Then, the errors of the models depending on the scale coversion method were calculated and compared. Equating errors were compared with Root Mean Squared Error (RMSE). In the study, the science test of the Trends in International Mathematics and Science Study (TIMSS) project administered in 2019 was used as the data collection tool. Models were examined according to the conditions as the the number of testlets, the number of independent items and the sample size. As a result of the study, it was observed that the error decreased in all models as the number of independent items and sample size in general. It was found that as the number of testlet increased, the error decreased in bifactor model and testlet response theory model, while the error increased in unidimensional item response theory model. In addition, it was concluded that the equating errors obtained from the Haebera and Stocking-Lord scale transformation methods were lower than the mean-mean and mean-sigma methods.