Standardizing assessment and evaluation practices in teaching Turkish as a foreign language

The primary objective of this research is to standardize the evaluation processes for instructors working in Turkish Teaching, Application, and Research Centres (TÖMER). TÖMER plays a crucial role in teaching Turkish as a foreign language, and this study aims to enhance the reliability and validity of the evaluation procedures. Additionally, the study aims to examine the measurement and evaluation practices implemented at TÖMERs to assess the current situation. In this research, designed according to the concurrent nested pattern of mixed research methods, qualitative and quantitative data were collected simultaneously, with an emphasis on qualitative data. The study involved 60 instructors divided into two groups. The first group (30 participants) followed a pre-test-post-test control group quasi-experimental design, with an experimental group (15 participants) and a control group (15 participants). The experimental group received 8 hours of standardized training, consisting of theoretical and practical aspects, and 2 hours of task-related processes. The participants' standardization levels in writing and speaking skills at A2, B1, and B2 levels were tested before and after the training using student products. The second group was used for qualitative data, and the opinions of relevant participants regarding the measurement, evaluation, and standardization processes at TÖMERs were gathered through written survey forms. Additionally, focus group interviews were conducted with 10 participants selected on a voluntary basis in groups of five to explore their perspectives further. To compare the examination structure applied at TÖMERs, a checklist was created based on the structure of the TOEFL exam, which is internationally recognized and has a similar purpose. Twenty-three Turkish proficiency exams were examined according to this checklist. The research utilized four different data collection tools: opinion forms, exam documents, hypothetical measurement and evaluation scenarios, and a checklist. Descriptive statistics, homogeneity and generalizability tests were employed for quantitative data analysis, while content analysis technique was used for qualitative data analysis. The examination results showed a significant increase in standardization levels for evaluating writing and speaking skills in the experimental group. The individual effect increased in both skills, leading to a decrease in the scorer effect. Moreover, it was observed that the pre-training and post-training scoring in the experimental group were heterogeneous in skills other than B1 level speaking, while no statistically significant differentiation occurred in the control group. According to the research findings, the processes of exam preparation in the examined TÖMERs were conducted in four different ways: relevant classroom teachers, administration and instructors, instructors, and measurement and evaluation unit/exam committee/R&D unit. There was no common understanding, and in some institutions, the exam control mechanism was not operational. Similar issues were encountered in the reading/evaluation of exams between institutions, where different exam evaluation processes were identified. Findings related to in-service training activities in TÖMERs revealed deficiencies in some institutions, while others showed that educational training activities such as educational materials and programs, innovative teaching approaches, experience sharing, and measurement and evaluation training were implemented. When examining the structures of exams applied at TÖMERs, it was observed that listening skills were not included in placement and mid-term exams or were considered as part of the speaking skill. Especially, depending on the implementation of the placement exam, writing and listening skills were not included in the exam. Evaluating Turkish proficiency exams at TÖMERs according to the checklist revealed that these exams were insufficient in presenting detailed result reports, international recognition, and including integrated speaking and writing tasks. However, it was determined that they showed similarity to TOEFL in terms of their general structure. Based on the research results, recommendations were made to practitioners, policymakers, institutions, and organizations. The research utilized four different data collection tools: opinion forms, exam documents, hypothetical measurement and evaluation scenarios, and a checklist. Descriptive statistics, homogeneity and generalizability tests were employed for quantitative data analysis, while content analysis technique was used for qualitative data analysis. The examination results showed a significant increase in standardization levels for evaluating writing and speaking skills in the experimental group. The individual effect increased in both skills, leading to a decrease in the scorer effect. Moreover, it was observed that the pre-training and post-training scoring in the experimental group were heterogeneous in skills other than B1 level speaking, while no statistically significant differentiation occurred in the control group. According to the research findings, the processes of exam preparation in the examined TÖMERs were conducted in four different ways: relevant classroom teachers, administration and instructors, instructors, and measurement and evaluation unit/exam committee/R&D unit. There was no common understanding, and in some institutions, the exam control mechanism was not operational. Similar issues were encountered in the reading/evaluation of exams between institutions, where different exam evaluation processes were identified. Findings related to in-service training activities in TÖMERs revealed deficiencies in some institutions, while others showed that educational training activities such as educational materials and programs, innovative teaching approaches, experience sharing, and measurement and evaluation training were implemented. When examining the structures of exams applied at TÖMERs, it was observed that listening skills were not included in placement and mid-term exams or were considered as part of the speaking skill. Especially, depending on the implementation of the placement exam, writing and listening skills were not included in the exam. Evaluating Turkish proficiency exams at TÖMERs according to the checklist revealed that these exams were insufficient in presenting detailed result reports, international recognition, and including integrated speaking and writing tasks. However, it was determined that they showed similarity to TOEFL in terms of their general structure. Based on the research results, recommendations were made to practitioners, policymakers, institutions, and organizations.