A machine learning approach to sentiment analysis: Insights from public perception of ai chatbots

In recent years, advancements in artificial intelligence (AI) and natural language processing (NLP) technologies have led to the widespread adoption of chatbots that interact with users. Large language models such as ChatGPT and Gemini AI are widely used in everyday interactions by a broad user base. However, understanding the public perception and user sentiment towards these models requires a comprehensive analysis. This study aims to analyze Twitter discussions about ChatGPT and Gemini AI to determine how these AI models are perceived by users. Two different sentiment analysis methods were compared those are rule-based sentiment analysis (SpaCy and TextBlob) and deep learning-based sentiment analysis (BERTweet). The results indicated that BERTweet outperformed rule-based approaches in sentiment classification, leading to its adoption as the reference model for further analysis. Subsequently, sentiment predictions were performed using various machine learning models, including Random Forest, Support Vector Machine (SVM), Logistic Regression, and LightGBM. These models were trained with three different embedding techniques which are DistilBERT, RoBERTa, and GloVe. The findings revealed that the combination of Logistic Regression with RoBERTa achieved the highest accuracy (81.8%). According to key findings of the study, the sentiment distribution of ChatGPT and Gemini AI differs significantly. ChatGPT received more negative sentiment, whereas Gemini AI was associated with a higher proportion of positive sentiment. The most common negative feedback for ChatGPT was related to 'information accuracy' and 'usage restrictions', while for Gemini AI, it was 'integration with the Google ecosystem' and 'reliability concerns' recieved more negative feedback. Word cloud and frequency analysis provided valuable insights into sentiment themes associated with both models. The findings of this study offer valuable insights into the development of AI-powered chatbots, enhancement of user satisfaction, and the design of next-generation AI systems. Future research may explore time-series analysis, multimodal sentiment analysis, and geographically segmented user sentiment trends for a deeper understanding of AI adoption and perception.