Investigation of different kappa coefficients for missing data stuation in educational research

In this study, it is aimed to examine how the Cohen Kappa, Gwet Kappa, Listwise Deletion Kappa, Fleiss Kappa, Light Kappa and Light Kappa methods based on Gwet Kappa, which are widely used in research to examine the agreement between raters in the case of missing data. In the study, for different scenarios obtained by assigning missing values with the MCAR method at different missing rates (5%, 10%, 15%, 20%, 25%, 30%), agreement tables were created for two or more rater cases, and the kappa coefficients were calculated. Comparisons were made according to the findings and it was examined which kappa type gave better results in case of missing value. It has been observed that when there are two raters, the number of participants is 10000 and the missing rate 30% Gwet Kappa coefficient gives better results. In case of missing data, Kappa based on Listwise deletion is a frequently preferred method, but it is recommended to use Gwet Kappa as it causes data loss. Since Gwet Kappa included the missing values in the calculation in a separate category, the results were more significant. In case of more than two raters, when the number of participants is 100, and the loss rates are 15% and 20%; when the number of participants is 1000 and the missing rate is 5%; When the number of participants was 10000, the loss rates were 5% and 10%, it was seen that the Light Kappa approach based on the Gwet Kappa suggested in the study performed better. According to the findings obtained from the real datasets, it can be said that the Light Kappa Based on Fleiss Kappa, Light Kappa and Gwet Kappa calculations for 3 raters, proposed within the scope of the study, are quite good in all missing rates as well.