Comparison of item discrimination parameter and reliability of an achievement test according to two theories

The study aimed to investigate the relationship between item discrimination indices and reliability coefficients estimated based on CTT and NIRT methods. In order to achieve this objective, the item discrimination indices of the items in the Measurement and Evaluation in Education test, which is a teacher-made achievement test and scored dichotomously, were estimated using CTT and NIRT methods. The study employed a descriptive research approach and a correlational research design. The study was conducted with a sample of 592 participants studying at Bolu Abant İzzet Baysal University. Item discrimination was examined using CTT, with the two-series correlation coefficient, and nonparametric item response theory, with Hi coefficients. Item reliability was examined using Cronbach's Alpha, equivalent halves reliability coefficient, MS (Molenaar- Sijtsma) and LCRC reliability coefficients. The analyses were conducted using the R package program (mokken 2.8.11) and SPSS 21.0 package program. The findings indicated a high and significant correlation between the item discrimination indices estimated based on the CTT and NIRT methods, as well as similar reliability coefficients. These findings suggest that the NIRT approach can be employed in scale development studies for ranking purposes, short tests, and small samples. The NIRT approach offers several advantages over the CTT approach. It is more data-oriented and exploratory, and it is not dependent on the group or items in the test. Therefore, it may be a more preferred approach for examining data in greater depth.