Regression based hyper parameter optimization for conversional neural networks

In Machine Learning problems, hyperparameters are adjustable parameters that affect the performance of the model. It is expected that users determine these hyperparameters. However, leaving the selection of these hyperparameters, which impact the performance of the model in terms of training and testing success, to the user poses a significant risk, especially in environments with a wide choice space. To address this issue, numerous methods have been developed for hyperparameter optimization. Some of these methods excel in terms of time, while others excel in accuracy criteria. Different methods may need to be employed for each dataset or problem. Additionally, well-performing methods might not be preferred due to their paid solutions. In this study, a hyperparameter optimization method has been developed that can yield results emphasizing both time and accuracy and is applicable to different datasets. The aim is to provide a method not only for researchers working solely on machine learning problems but also for all problem solvers with parameters that need optimization. This developed method has been compared with two established methods: random search and Bayesian optimization. Furthermore, it has been tested on three different datasets CIFAR-10, FASHIONMNIST, and MNIST. The results indicate its superiority over random search and Bayesian optimization.