Determination of variables affecting user experince in CAT application and their possible effects

This study was conducted to determine the variables affecting user experience in computer adaptive tests and their possible effects. The study was conducted within the framework of a convergent parallel mixed methods design. The qualitative dimension of the study involved examining student opinions using content analysis methods, while the quantitative dimension was conducted as a survey study. In addition, the study demonstrates its R&D characteristics with its contributions to CAT designs. In this direction, 130 test items related to the measurement and evaluation course in education were written to create the item pool. Based on the opinions of subject and language experts, 40 items were removed from the pool, and 90 items were administered to individuals. The applications were conducted in two different ways: an incomplete test pattern and the application of all items. Fifteen items that violated the principle of local independence, one of the assumptions of Item Response Theory, and had a low item fit index were also removed from the test. Post-hoc simulations were conducted based on data collected from 293 education faculty students for the final pool of 75 items. According to the simulation results, the Rasch model was determined to be the most appropriate measurement method and value for the IRT model, with a standard test length of 10 items as the stopping rule; Maximum a Posteriori for ability estimation and a randomly selected item from the range of -0.5 to +0.5 for the initial level. Four CAT's were designed based on the scope and duration of feedback variables that were thought to affect the CAT user experience. Individuals participating in the CAT application were assigned to different tests via a website. A total of 131 education faculty students participated in the application, and their opinions regarding the CAT experience were collected after the application. As a result of the CAT application, it was observed that individuals who took tests with extensive feedback had higher confidence and motivation in the test. Although the opinions of individuals who did not see the time information varied, individuals generally iv reported lower stress levels. In addition, individuals' opinions about the CAT application and system were examined. In conclusion, it was observed that comprehensive and clear feedback improves the user experience