Hybrid approach for hyper-parameter selection in deep neural networks with statistical and machine learning

In the literature, a very important problem in deep neural networks is the determination of hyper-parameters. There are many combinations that can be tried for the selection of hyper parameters to be used in data analysis. It is not possible to try all possible combinations, especially when it comes to big data. Although there are no general rules for making the selection specified in the literature, there are some suggested and used approaches. On the other hand, since ANN is a data-driven method, it is often not possible or applicable to use an approach, which is introduced for a specific data, for another data. In 2019, Aladag proposed a hybrid approach based on statistical and machine learning for the solution of the architectural selection problem in ANN. This issue is one of the most important problems of ANN. In his hybrid approach, the structure of the relationship between ANN architectures and predictive performance can be analyzed using a simple linear regression model. In this way, the determination of architecture is performed statistically and the relationship structure between the number of neurons used in the architecture and the prediction performance can be revealed. The approach proposed by Aladag provides two very important advantages. First of all, architecture selection can be done statistically and systematically according to the analyzed data. Secondly, the performance of many architectures can be statistically predicted even without using these architectures since the relationship structure model between the architectures and the prediction performance is obtained. In the thesis study,iv the hybrid approach proposed by Aladag 2019 for ANN architecture selection was used by applying both linear and nonlinear regression analysis to the hyper-parameter selection in deep neural networks. The hybrid approach was applied to the diabetes (diabetes.csv) data. As a result of the application, it has been observed that the hybrid approach gives very good results.