Comparison of confirmatory factor analysis estimation methods for mixed-format tests

The purpose of this study is to compare maximum likelihood (ML), robust maximum likelihood (MLR), unweighted least squares mean-and-variance adjusted (ULSMV), weighted least squares (WLS), weighted least squares mean-and-variance adjusted (WLSMV) and Bayes estimation methods used in confirmatory factor analysis (CFA) according to simulation conditions determined within mixed format tests. Percentage of open-ended items (10%, 20%, 40%, 50%), distribution of open ended items (normal, right skewed, left skewed), number of score categories of open ended items (3, 4 and 5), mean factor loading (0.40, 0.60, 0.80), sample size (200, 500 and 1000) and the number of items in the test (20, 30, 40) are specified within Monte Carlo simulation conditions. 1000 replication was used for each condition. Unidimensional constructs are investigated. Convergence rate, improper solutions, percentage of correct estimate (PCE), relative bias (RB) and relative bias of standard error (SHY) values obtained from the estimation methods in the study are considered as dependent variables. In addition, the findings obtained from the implementation of Monitoring and Evaluation of Academic Skills survey which was conducted by the Republic of Turkey Ministry of National Education (MoNE) in 2016 was compared with the simulation study. As a result, it was observed that estimation methods make more accurate estimates with increasing sample size and average factor loadings. It can be said that there are no estimation methods delivering the best performance for all conditions. However, when all the conditions were evaluated, the performance of ULSMV method was better than the others. WLSMV was similar to ULSMV. In addition, it was observed that Bayes method yielded better results in some conditions in small samples than ULSMV and WLSMV.