Adaptive learning rate and the effect of complex numbers on machine learning

The learning coefficient is one of the fundamental hyperparameters that directly affects the training performance of artificial neural networks. An improper selection of this parameter can negatively impact the network's convergence rate, generalization ability, and overall training process. Although adaptive learning coefficients, error-based dynamic adjustments, and layer-based methods have been proposed in the literature to solve this problem, a precise and universal solution has not yet been developed. To overcome the limitations arising from the use of a fixed learning coefficient, the KarcıFANN method is employed. Due to its fractional-order derivative structure, the KarcıFANN method can produce complex values in certain cases. Processing only the real components during training can lead to information loss and optimization instabilities. In this thesis, the effect of using complex values on learning in artificial neural networks (ANNs) is examined. Accordingly, a method that processes real and imaginary components together is proposed.In the classification process carried out using the Digits, MNIST Rotation 45°, 2BFT-Digits, and 2BFT-MNIST Rotation 45° datasets, three models were used: the Real model, which uses only real values, the Complex model, which uses both real and imaginary components together, and the Abs model, which evaluates only the magnitude information of the complex values. The results showed that the Complex model achieved the highest performance, while the Real model exhibited lower performance on frequency-transformed data. Moreover, since the Abs model does not utilize phase information, its performance was found to decrease. These findings demonstrate that computations using complex numbers significantly enhance the learning performance of the KarcıFANN method. The performances of KarcıFANN and classical ANN methods were comparatively evaluated on the Kuzushiji-MNIST, GinaPrior2, and Sign-MNIST datasets. In addition, the convergence behaviors of KarcıFANN with SGD, Momentum-based GD, and Adam optimization methods were examined using the XOR problem. Finally, the effects of using different loss functions (MSE, RMSE, and MAE) with KarcıFANN on model performance were analyzed. Keywords: KarcıFANN, artificial neural networks, SGD, momentum-based GD, ADAM