Examination of the construct validity of Likert type scales with data mining, artificial neural networks, factor analysis and Rasch models

Data collection instruments which are important building blocks of measurement and evaluation activities are expected to comply with the standards. Therefore, one of the issues that need close attention in developing measurement instruments with the most possible relevant psychometric properties in the test development process is the selection of items to be included. In this study, it was aimed to compare the construct validity evidence of the forms that were obtained through exploratory factor analysis, artificial neural networks, data mining and Rasch rating scale model in order to reveal the item selection and construct validity in the scale development process. Within the scope of the research, the data were collected by applying the "Statistical Attitude Scale" pilot form, which was used in a previous study, to individuals enrolled in undergraduate and graduate programs, and individuals who already graduated and not currently enrolled in a graduate program. The data were analyzed using exploratory factor analysis, classification and regression trees, self-organizing mapping and Rasch rating scale model. As a result of the analyses, it was seen that the number of dimensions and the distribution of the items to the dimensions could vary in different methods. The fit indices obtained from the scales that consisted of the items determined through different methods were compared using confirmatory factor analysis. The research results showed that self-organizing mapping analysis was useful in terms of selecting items that would increase the fit values in confirmatory factor analysis, while the classification and regression tree methods were impractical. Additionally, it was seen that the confirmatory factor analysis fit indices of the form created using the Rasch rating scale model were also sufficient and revealed detailed results about the items and the individual.