Comparison of the results of predicting the academic achievements of high school students with machine learning models using ChatGPT and KNIME

Rapid developments in artificial intelligence technologies are radically transforming not only daily life but also data-driven decision-making in education. This change requires teachers to effectively analyze large volumes of student data and improve learning environments based on evidence. The purpose of this study is to comparatively examine the performance of the generative artificial intelligence tool ChatGPT and the traditional open-source platform KNIME in predicting high school students' academic achievement through machine learning. The study was designed with a quantitative correlational survey approach and operated based on the CRISP-DM process. Data were obtained from publicly available data consisting of 33 academic and socio-demographic variables for two Portuguese subjects (Mathematics n = 395 | Portuguese n = 649). Five supervised algorithms (KA, RF, NB, KNN, SVM) were applied in four scenarios (A - all variables; B - excluding Note2; C - excluding Note1; D - excluding Note1 and Note2) for both binary and multiclassing on both instruments, and accuracy, recall and F1 scores were compared to the ZeroR baseline. Results; (i) KNIME outperformed ChatGPT by 0.07-0.15 points in binary classification with an average F1 = 0.92→0.85, and by ~0.07 points in micro-F1 in multiclass classification; (ii) Removing academic grade variables led to a 10-15 point decrease in accuracy, while family relationship with absenteeism was prominent in KNIME models and parental education level was prominent in ChatGPT models. In conclusion, it is possible to predict academic achievement based on academic and socio-demographic data. Generative artificial intelligence tools, while currently underperforming data mining software, can play a complementary role, providing early discovery and speed gains. This study presents findings in the field of educational data analytics that provide guidance for artificial intelligence professionals, policy makers and future research.