Investigation of adaptive multistage test based on test assembly methods

In this research, the performance of top-down and bottom-up test assembly methods were compared according to the sample size, panel design and module length of the Adaptive Multistage Tests (MST). MSTs formed according to the both test assembly methods were also evaluated in terms of the classification test. Within the context of the research, data for PISA 2015 were utilized and a simulation study was conducted according to the parameters predicted based on these data. Item parameters calibrated according to the 2 PL model from the two-category data set obtained the PISA data applied in the year 2015 comprised the item pool of the research. Of the research variables, module length variables 6 and 12, panel design variable "1-2", "1-2-2" and "1-2-3", and the sample size variables 250 and 2000 were determined in the study. Analysis results for each condition were compared in terms of mean error, bias and classification accuracy. According to the results obtained from the MST simulation formed based on top-down test assembly method, it was determined that the mean error value decreased when the module length of MST applied to the small and large samples increased. The mean error values decreased in case of the transition from "1-2" panel design to panel design "1-2-2" and "1-2-3" took place. The increase in the sample size lead to a certain amount of decrease in the mean error values of "1-2" and "1-2-2" panel design in short and moderate module lengths. The increase in the module length lowered the bias values of the panel design in both sample types. The transition of panel design from "1-2" to "1-2-2" and "1-2-3" decreased the mean bias values in both sample sizes and module lengths. The increase in sample size also reduced the bias slightly for all conditions. Individuals who filled the MST were compared with the passing ratios 70%, 50% and 30% respectively and the high classification ratios and Cohen's Kappa coefficient of agreement. According to the results obtained from the MST simulation constructed according to bottom-up test assembly method, the mean error value decreased as the module length and panel design in MSTs applied to small and large samples increased. The increase in the sample size lead to a decrease in the mean error values of "1-2" panel design in short and moderate module lengths, "1-2-2" panel design in short module length and "1-2-3" panel design in moderate module length. The increase in sample size and module length decreased the bias values of panel design "1-2", "1-2-2" and "1-2-3" applied to the small and large samples. It was found that MSTs consisting of panel design "1-2", "1-2-2" and "1-2-3" that contain short and moderate length modules applied to both samples classified the individuals at a high rate at 70%, 50% and 30% passing ratios.