Logarithmic learning differential Convolutional Neural Network

Convolutional Neural Networks (CNNs) have been instrumental in transforming the field of computer vision by their innovative design and training methodologies for image classification. The differential convolutional neural network with simultaneous multidimensional filter realization has been successful in improving the performance of the convolutional neural network, although it suffers from calculation cost drawbacks. This Thesis proposes a solution to the drawback by introducing logarithmic learning integration into the differential Convolutional neural network. The proposed approach employs LogRelu activation, a logarithmic cost function, and a unique logarithmic learning method for faster error minimization and convergence. The study evaluates the effectiveness of these proposed methods across multiple datasets and optimization algorithms, including SGD and Adam. The experimental findings show that integrating LogRelu leads to performance improvements ranging from 1.61% to 5.44% across convolutional neural networks, while the same integration on ResNet-18, ResNet-34, and ResNet-50 enhances top-1 accuracy in the range of 3.07% and 9.96%. Moreover, the Logarithmic Differential CNN consistently outperforms standard CNNs with an accuracy increase of up to 3.02% with the adaptation of the Logarithmic Cost Function. The study also evaluates various activation functions on Differential CNN models and pre-trained models using MNIST and Cifar10 datasets. Among the activations, LeakyReLU, ELU, SELU, and LogRelu consistently outperform ReLU, with LogRelu being particularly effective at lower learning rates. Although the differential CNN faces compatibility issues with certain optimizers, it displays adaptability and excels with an SGD rate of 0.01 and lower rates for Adam and RMSprop.The experimental results proved the efficiency of the proposed approach.