Artificial intelligence-based approaches for anomaly detection and transfer in retail data in the data warehouse

This study presents several recommendations for anomaly detection based on source data taken into the data warehouse. Efforts have been made to provide information through the testing of certain artificial intelligence algorithms to prevent the entry of dirty and noisy data into the data warehouse. Having clean and healthy data in the data warehouse is quite beneficial for areas such as data science studies. The current chip crisis has increased the importance of savings in physical machines. In this context, the proposed approach has the potential to provide savings in physical machines. LSTM, Isolation Forest, Local Outlier Factor, and SVM algorithms have been tested. Considering that the LSTM algorithm may be temporally disadvantaged with large data, the Local Outlier Factor and SVM algorithms have also been included in the evaluation. A study has been conducted among the Isolation Forest, LSTM, Local Outlier Factor, and SVM algorithms. However, it has been observed that each algorithm detects different anomalies, which indicates that the "single best algorithm" approach is insufficient for anomaly detection problems. Consequently, the integration of algorithms as a hybrid ensemble model carries significant potential for enhancing performance by leveraging the strengths and balancing the weaknesses of various algorithms. The applicability and effectiveness of this approach should be validated with comprehensive experiments on larger datasets and evaluated with a cost-benefit analysis according to specific application scenarios.