Variable selection with missing data imputation: An analysis of machine learning approaches

Missing data is one of the most fundamental challenges frequently encountered in statistical analysis processes, directly threatening the reliability of results. In this thesis, the performance of traditional and machine learning-based imputation methods under three missing data mechanisms - Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) - and four missing rates (5%, 10%, 20%, and 40%) has been systematically compared within a Monte Carlo simulation framework; the reflections of the variable selection process on this comparison have also been examined. Simulation datasets containing sample sizes of n = 50, 500, 1000, and p = 5, 10, 20 independent variables were used in the study. Independent variables were generated from a multivariate normal distribution within a compound symmetry correlation structure (ρᵢⱼ = 0.3), with 3, 6, or 12 relevant variables included in the true model depending on the value of p. The imputation methods examined include mean, median, k-Nearest Neighbors (kNN), Multiple Imputation by Chained Equations (MICE), and Random Forest-based missForest. In the variable selection phase, LASSO (Least Absolute Shrinkage and Selection Operator), Random Forest Variable Importance, and the Boruta algorithm were compared. One hundred Monte Carlo replications were performed for each scenario, and performance was evaluated using Normalized Root Mean Square Error (NRMSE), F1 score, True Positive Rate (TPR), False Discovery Rate (FDR), and the Jaccard stability index. Simulation results revealed that the MICE method consistently produced the lowest imputation error across all missing mechanisms and sample sizes (NRMSE = 0.574– 0.904). Mean and median imputation exhibited the lowest performance under all conditions; under the MNAR mechanism, NRMSE values substantially exceeded the 1.0 threshold. As sample size increased, the imputation accuracy of all methods improved; under n = 1000, MICE achieved NRMSE = 0.574 under the MCAR mechanism. In terms of variable selection, LASSO demonstrated the most consistent performance for small samples with n = 50, while Boruta produced competitive results with LASSO for larger samples with n ≥ 500. The MICE + LASSO combination emerged as the most balanced imputation–variable selection approach across all conditions. This thesis revealed that the type of missing data mechanism and sample size play a decisive role in the selection of imputation method; the MICE + LASSO combination is appropriate for all mechanisms, while the MICE + Boruta combination offers a strong alternative for larger samples. The findings also emphasize that imputation quality directly determines variable selection success and that the two processes must be planned jointly.