Hyperparameter optimization of support vector machines with a new algorithm combining grid search and particle swarm optimization

Hyperparameter optimization is vital in improving the prediction accuracy of Support Vector Machines (SVM) as in all machine learning algorithms. This study introduces a new hybrid optimization algorithm, namely PSOGS, which consolidates two strong and widely used algorithms; Particle Swarm Optimization (PSO) and Grid Search (GS). This hybrid algorithm experimented on twelve benchmark datasets. The speed of PSOGS and the prediction accuracy of PSOGS-optimized SVM models (PSOGS-SVM) were compared to those of its constituent algorithms (PSO and GS) and another hybrid optimization algorithm (PSOGSA) that combines PSO and Gravitational Search Algorithm (GSA). The prediction accuracies were evaluated and compared in terms of Accuracy and F-Score for classification problems and Root Mean Square Error, Mean Absolute Percentage Error, and Multiple Correlation Coefficient for regression problems. For the sake of reliability, the results of the experiments were obtained by performing 10-fold cross-validation on 30 runs. The results showed that PSOGS yields comparable prediction accuracy with GS, performs much faster than GS, provides slightly better results with less execution time than PSO. Besides, PSOGS-SVM presents more effective results than PSOGSA-SVM in terms of both prediction accuracy and execution time. As a result, this study proved that PSOGS is a fast, stable, efficient, and reliable algorithm for optimizing hyperparameters of SVM.