The effect of missing data handling methods on differential item functioning with testlet data

The purpose of this study was to examine the effect of missing data handling methods on differential item functioning (DIF) with testlet data. The study was conducted on two different data sets consisting of six testlets which contain 20 reading comprehension items of a foreign language test. Data with left-skewed distribution was referred to as data1 and data with right-skewed distribution was referred to as data2. Under missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) missing mechanisms, the effect of listwise deletion (LD), zero imputation (ZI) and fractional hot-deck imputation (FHDI) methods on the DIF detection performance was investigated. A bifactor multidimensional item response theory model for testlets with covariates was used as the DIF detection method. Two sample size (1000 and 2000) and two missing data percentage (5% and 15%) conditions were the other conditions examined in the study. Results of the study indicated that examining the correlation between DIF values obtained from both data sets under all conditions and DIF values obtained from complete data sets, LD method had the lowest correlations. Besides, in all conditions correlation values decreased with the increase of missing data percentage regardless of the missing data handling method used. As a result of the DIF analyses from data1, it was concluded that performance of detecting DIF-free items was similar with three missing data handling methods at 5% missing data percentage condition. In eight conditions conducted on data2, it was found that there was recovery in the performance of detecting DIF-free items as the sample size increased; in other conditions all DIF-free items were identified correctly for both sample size. Performance of detecting DIF items with ZI and FHDI methods under MCAR mechanism at 15% missing data percentage was found to be identical.