An overview of some mathematical techniques in deep learning

In this thesis, we will explore the mathematical aspects of deep learning. Our goal is to analyze the mathematical tools used in deep learning. We aim to understand mathematical challenges such as minimizing convex cost functions. Firstly, by taking a brief look at the history of deep learning, we start to gain a better understanding of this field. Then, we delve into the principles, applications, and workings of deep learning and artificial neural networks. Next, we introduce activation functions that allow neural networks to depart from linearity and help them learn more complex relationships. After discussing sigmoid and ReLU functions and their advantages, we create a simple neural network and learn how to mathematically compute the weights and biases of neural networks. Subsequently, we work with a handwritten example to understand how artificial neural networks function from a mathematical perspective. We learn to compute stochastic gradient descent to minimize the cost function and apply this method to train the network. Then, we apply the backpropagation method to determine the contribution of each weight and bias term to the error. This thesis also includes MATLAB code implementing backpropagation and stochastic gradient descent, along with an image classification example. Additionally, we explore the three main types of layers in convolutional neural networks, understand the principles of convolutional layers, perform calculations, and use Sobel filters. Finally, we conclude our thesis with an image classification example using the Keras CIFAR-10 dataset. In this example, by examining all layers of the neural network in detail, we understand its functionality and how it reaches the training stage.