Examination of the PISA 2018 reading skills sub-test with different methods in terms of differantial item function

This study aims to examine whether the items in the reading skills subtest in the PISA 2018 implementation show differential item functions according to different culture and language variables. Within the scope of this purpose, from the item packages in the PISA 2018 reading skills subtest, the 1st package for the Basic section (Core RC1), the 1st package for the 1st Stage (Stage 1 - R11H), and the 1st package for the 2nd stage (Stage 2 - R21H) were selected. The data set of the study consists of the answers given by 536 participants from Turkey, France, and the United Kingdom who answered these substance packages. Before proceeding to the DMF analysis, the data set was examined in terms of missing values and outliers. In addition, descriptive statistics were used to determine the general characteristics of the group distribution. The number of items, number of students, mean, standard deviation, skewness, kurtosis, highest score, lowest score, and Cronbach Alpha values were calculated for all subgroups separately. Mantel-Haenszel (MH), Logistic Regression (LR), and SIBTEST (Simultaneous Item Bias Test) methods were used to perform DIF analyzes of the items under consideration. Among the selected item packages, only 35 items that could be scored in two categories were included in the DIF analysis. Countries in the study group; analyzed in pairs: Turkey – UK, Turkey – France, and France – UK. As a result of the analyzes carried out; in the Turkey - United Kingdom group, 14 items with the MH method, 18 items with the LR method, and 11 items with the SIBTEST method were identified as having DIF. In the Turkey - France group, six items with the MH method, eight items with the LR method, and five items with SIBTEST method were determined as having DIF. In the France - United Kingdom group, nine items with the MH method, thirteen items with the LR method, and eight items with the SIBTEST method were identified as having DIF. Finally, it was examined to what extent the results of the DIF determination methods used were in agreement with each other in terms of the items which having DIF. In general, although the number of items identified by the methods as having DIF is different, it has been found that the items identified as having DIF are mostly similar to each other. When the calculated Spearman-Brown rank difference correlation coefficient was examined, it was concluded that the highest agreement was seen between the MH - LR methods, while the lowest agreement was observed with SIBTEST.