The impact of natural language processing based sentiment analysis and variable selection methods on stock price prediction

The aim of this study is to examine the impact of news texts on stock price prediction. In addition, the impact of dimensionality reduction techniques and regularization methods on prediction performance has been examined. For this purpose, hybrid variations of dimension reduction techniques and regularization methods, along with relevant techniques, have been used. Four different natural language processing variables have been calculated from the news articles and incorporated into the models. Natural language processing variables were combined with time series variables in various combinations, and their effects were comparatively analyzed with the forecasting performance of ARIMAX, ANN, LSTM, and GRU methods. Applications were carried out on simulation studies and eight stocks. The effects of dimension reduction techniques, regularization methods, and the integration of natural language processing variables in the model have been analyzed through applications. The findings have shown that not only technical indicators but also news text analysis positively contribute to the accuracy of stock price predictions. The combination of dimensionality reduction techniques with regularization methods has been effective in managing variable complexity. By utilizing a hybrid approach and news sentiment analysis, a comprehensive financial market forecasting framework has been provided.