Comparison of artificial neural networks using different activation functions with conventional regression analysis

Artificial intelligence is a system that imitates human intelligence, takes decisions by making inferences and sense of the information it collects, and puts it into practice. Artificial Neural Networks (ANNs) are subfield of artificial intelligence and widely used by researchers. At the basis of every ANN lies an activation function (AF) which plays an important role in the success of training networks. AF creates a mathematical link between the input and output of the neuron and adds non-linearity to the model. In addition, it contributes significantly to the speed of the network and its ability to predict accurately. The aim of this thesis is to define new AFs that combine the advantages of predefined AFs and outperform them. For this purpose, Gen-Swish, Mean-Swish, ReLU-Swish, TS-Swish, SiPA, TS-Sigmoid, Exp-Swish, Sinc-Sigmoid AF were defined by using Sigmoid, ReLU, Swish and algebraic AFs. The results obtained using the proposed new AFs in different network architectures and conventional regression results were compared and it was seen that ANN models were more successful than regression. In addition, although the network performances vary depending on the data sets used, it has been determined that some of the suggested functions are more successful than the mostly used functions.