Investigation of accuracy of ability estimations and classification indices under cumulative, ideal point and mixed model with bayesian and nonbayesian methods

The aim of this study is to examine the effect of modeling item response processes on accuracy of ability estimations in the framework of bayesian and nonbayesian methods, and on classification accuracy and classification consistency at different cut-off points along the ability scale based on these ability estimations. In this direction, GRM (Samejima, 1969), which is one of the cumulative models, and GGUM (Roberts, Donoghue & Laughlin, 2000), one of the ideal point models, were used in modeling the response processes. In addition to these models, a mixed model was created with the combination of cumulative and ideal point models based on the deviation of delta parameters (δ) from the distribution mean. MAP and EAP as bayesian methods with WLE as nonbayesian method were used for the ability estimations. Based on the models and methods, accuracy of ability estimations were investigated through the two-way repeated measures ANOVA. After the investigations of ability estimations, classification indices were investigated at the cut scores located on both lower and higher parts of the theta scales in terms of three way interaction effect of cut score, model and method. Accordingly, variation of the classification indices in terms of model and method separately on three cut-off point pairs c (0,10, 0,90), c (0,15, 0,85) and c (0,20, 0,80) approaching from the extreme to the middle on the ability scale was examined and statistical differences were tested through three-factor repeated measures ANOVAs. When the findings obtained from the research are examined, it is observed that the accuracy of ability estimates differs depending on the interaction of the model and method used. Highest accuracy of ability estimates is obtained under GGUM and it is followed by mixed model. It is observed that under the GGGUM and mixed model, among the bayesian and nonbayesian methods MAP provides the most accurate results and it is followed by WLE. Then, it is concluded that EAP results in higher errors when the focus is estimation of nonmonotonic items. However, under the GRM, a cumulative model, the order of the methods is as MAP, EAP and WLE which points out that bayesian methods provides higher accuracy than nonbayesian method. After examining the ability estimates, classification indices obtained based on these estimates were examined. According to the results obtained from the examinations for the classification indices, it is observed that interaction of model and method at cut scores determined on lower and upper parts of ability scale, and the effect at the extreme locations is decreasing with moving toward the middle part of the ability scale. Investigations are conducted in order to reveal at which level of the cut score location interaction effect of model and method is observed. Depending on the results, it is observed that interaction effects of model and method are likely to occur at the upper part of ability scale, and models and methods provides similar results at the lower part of ability scale.When examining in terms of models, it is observed that GGUM gives better results than mixed model and GRM at the upper end, and GGUM is followed by mixed model. When examined in terms of methods, it is observed that the effectiveness of the methods depends on the type of model selected. Although WLE, MAP and EAP generally provide similar results under GRM, under GGUM and mixed model EAP results in lower classification indices than WLE and MAP, and WLE and MAP were observed to result in similar and higher classification indices. Finally, it is observed that while MAP stands out from bayesian methods in the accuracy of ability estimates, WLE, one of the nonbayesian methods, stands out in classification indices. The findings obtained from the research were discussed in relation to possible situations that may be encountered in real data, and suggestions for the use of models and methods for possible measurement situations to be encountered in practice are presented.