Classification of environmental attitudes with machine learning algorithms: An engineering approach based on feature selection

Nowadays, the performance of machine learning (ML) models in data-driven decision-making processes largely depends on the quality of the used data and the selection of meaningful features. Particularly in high-dimensional datasets, the presence of redundant or noisy feautres impedes the learning process, but also induces overfitting and diminishes the interpretability of model outputs. Therefore, feature selection is regarded as a critical preprocessing step that not only reduces computational complexity but also improves fundamental performance metrics, such as accuracy and generalizability. Studies in the literature show that identifying optimal feature subsets improves classification performance and reduces training duration.In this thesis, the effects of different feature selection methods on the classification performance of various ML algorithms are systematically examined. As a case study, an open-access survey dataset containing individuals' environmental attitude levels with a multivariate structure has been used. To identify the most significant features in this dataset, ten different methods have been applied, including Analysis of Variance, Chi-square test, Random Forest (RF), LASSO (Least Absolute Shrinkage and Selection Operator), Principal Component Analysis (PCA), Voting, Recursive Feature Elimination, Mutual Information, and Variance Threshold. Using the selected feature sets, ten different classification models have been trained, including Naive Bayes, Gradient Boosting, k-Nearest Neighbors, Support Vector Machines, Logistic Regression, Multilayer Perceptrons (MLP), Bagging, and Linear Discriminant Analysis. During the modelling process, the dataset has been divided into 80% training and 20% testing, and the performance evaluation has been conducted using 5-fold cross-validation method. The findings reveal that feature selection methods based on PCA and RF, when combined with MLP algorithm, has been achieved the highest performance with an accuracy rate of 98.7%. Additionally, LASSO- and Voting-based approaches have also demonstrated remarkable performance. Overall, it has been observed that the appropriate matching of feature selection methods with classification algorithms has statistically significant effects on model performance.In conclusion, this thesis extensively investigates the effects of feature selection techniques on ML based classification models for environmental attitude data. The findings reveal that feature selection plays a critical role in terms of model performance in classification problems for high-dimensional datasets; accordingly, the study makes significant contributions to the literature at methodological and practical levels. The results, supported by numerical analyses, emphasise the importance of feature selection in the model optimisation process and suggest a flexible framework that can be applied to different data types and problem domains.