An alternative approach in measuring computational thinking: Performance-based measurement

Computational thinking skills are becoming part of the individual qualities needed for today's world. As well as determining the qualities of the individual needed, teaching and measuring are among the important issues that have been discussed. For this reason, the aim of this study is to develop an online, interactive, valid, reliable and useful performance-based measurement tool for measuring computational thinking skills. The research was conducted by using the descriptive survey model. Therefore, quantitative data was collected by using an interactive web application which is a performance-based measurement tool developed, a skill test consisting of questions equivalent, and a valid and reliable psychometric scale. The validity and reliability analyses of the web application were made with data obtained from the psychometric scale and skill test used in the research. The questions in the skills test are prepared to have the same properties as the questions prepared for the online platform. Although the questions in the web application and the questions in the skills test are completely different from each other, they are expected to measure the same skills. The developed performance-based measurement tool has 12 questions covering 7 different skills. The questions can be answered in different ways according to the answer expected from the participant. In other words, the participants are required to perform a number of interactive operations such as drag-and-drop, flip, and mark during the question answering phase. At the stage of preparing the questions, expert opinions and sources obtained from the literature took an important position. Each of the questions, consisting of different difficulty levels, was automatically evaluated over 10 points. Therefore, a participant who correctly answers all the questions in the developed performance-based measurement tool can receive a maximum of 120 points. Besides, the data collection process consists of two stages. In the first phase, 156 high school students completed statements on the psychometric scale and then answered the questions on the web app which is a performance-based measurement tool. In the second phase, 47 high school students answered multiple-choice questions in the skill test in the form of paper and pencil immediately after completing the psychometric scale. At the same time, the students answered 12 questions in the web application. In other words, the study group consists of 203 high school students. In this context, statistical analyses were carried out with data obtained from three different data collection tools. As a result of the normality analysis conducted on the obtained data, it was determined that the data showed a normal distribution. However, since the number of samples in each category included in the second stage did not meet the assumptions in parametric tests, non-parametric tests were used. Independent Sample T-test, One-Way ANOVA test, and Tukey HSD test were used for parametric tests. For non-parametric tests, Kruskal-Wallis and Mann-Whitney U tests were used. Besides, Velicer's MAP Test and Horn's Parallel Analysis Test were used for exploratory factor analysis of the developed interactive web application. Cronbach's Alpha test and Spearman-Brown test were applied for reliability studies. In the context of convergent validity, a bivariate correlation test was used. As a result of all the analyses, a valid and reliable performance-based online platform has been developed for measuring computational thinking skills. According to the results obtained in the analyses, it was found that the computational thinking skill levels of high school students in the study group were medium-level. Besides, students' computational thinking levels were examined based on gender, class level and school type variables. As a result of the examination, students' computational thinking skill levels did not differ by gender in the context of the web application and skill test tools, while the results obtained from the psychometric scale differed significantly in favor of male students. While no significant differences were found in terms of psychometric scale and web application when examined specifically for class levels, significant differences were found in favor of 12th grade in the data collected from the skill test. Finally, as a result of analyses conducted in the context of the school type, significant differences were observed for the web application and skill test measurement tools, while no differences were found for the psychometric scale by school type. It has also been concluded that the performance-based measurement tool makes a better measurement than the psychometric scale.