A comparison of Angoff and direct consensus standard setting methods

In this study, Angoff and Direct Consensus standard-setting methods were compared in terms of the cut-off scores they produced and the opinions of the experts on the methods. The student data of the study were obtained by applying a 24-item achievement test for the "Numbers and Operations" of 5th grade mathematics to 52 students. 10 experts were employed to implement and evaluate the methods. The cut-off point for the Angoff method was calculated as 52.04, Direct Consensus method was calculated as 49.17. According to the Angoff method 52%, according to the Direct Consensus method 58% of the students were considered successful. The difference between the percentage values was analized with the z-test and it was not significant. According to the methods, a perfect compatibility and a high correlation were determined by calculating Cohen's Kappa coefficient of 0.88 and Spearman's coefficient of 0.89 between students' successful-unsuccessful states. The relationship between the minimum passing scores (MPS) according to the methods was investigated with the Pearson's r, and the value was 0.80, and it was concluded that the MPSs were positively correlated. When the opinions of the experts on the methods were analyzed descriptively, it was concluded that a higher positive opinion was developed for the Direct Consensus method. KEYWORDS: Standard setting, Cut-off Score, Angoff Method, Direct Consensus Method