An investigation of ai chatbots on EFL high-school learners' speaking proficiency, anxiety, motivation and digital literacy

This study investigates the impact of AI-driven interaction on English as a Foreign Language (EFL) learners' speaking proficiency, speaking anxiety, motivation, and digital literacy. With the rapid advancements in artificial intelligence, AI-based conversational agents have emerged as potential tools to enhance oral communication skills in language learning. This research employs a quasi-experimental design with a mixed-methods approach, combining quantitative and qualitative data collection techniques. The study involves 60 high school 11th-grade EFL students, divided into an experimental group (n=30) utilizing AI-powered conversational (ChatGPT and Google Gemini) for speaking practice, and a control group (n=30) engaged in traditional teacher-guided speaking activities. Participants were selected from similar classes. Over a ten-week period, the experimental group engaged in AI-driven speaking tasks, shared their recordings via Google Classroom, and received feedback from both AI and teacher. The control group continued with the regular coursebook-based speaking activities under teacher supervision. Data collection included pre- and post-test assessments of speaking proficiency (evaluated using TOEFL iBT rubrics), speaking anxiety (measured using the FLSAS Questionnaire by Kayaoğlu & Sağlamel, 2013), and digital literacy (assessed using the scale by Hamutoğlu et al., 2017). Additionally, students' motivation levels were examined through Schmidt et al.'s (1996) language learning motivation scale. Qualitative data were obtained through semi-structured interviews with the experimental group and classroom observations. Quantitative analysis was conducted using paired-samples and independent samples t-tests to measure within-group and between-group differences. ANOVA was employed to assess statistical significance among variables, and correlation analyses were performed to examine relationships between speaking proficiency, motivation, and anxiety. Reliability analyses (Cronbach's Alpha) ensured internal consistency of the instruments. Qualitative data were analyzed through thematic and descriptive analysis to categorize students' perceptions and classroom interactions. Findings indicate that AI-driven speaking practice significantly improved students' speaking achievement scores compared to traditional instruction. Moreover, AI-integrated learning contributed to lower speaking anxiety and increased motivation for language learning. Qualitative data highlighted students' positive attitudes towards AI tools, particularly in receiving immediate feedback and engaging in autonomous learning. The results underscore the pedagogical value of AI-driven tools in supporting self-regulated speaking practice and enhancing digital literacy. This study provides empirical evidence on the role of AI in language education, offering insights for integrating AI technologies into EFL curricula. Future research may examine the long-term effects of AI-based speaking interventions and their impact on learners with varying proficiency levels.