Comparison of deep learning and Bayesian deep learning methods for image classification

When constructing network architecture in basic convolutional deep learning, it is necessary to determine the depth of the network which specifies the number of inner layers, and the initial values of learning parameters such as learning rate, momentum and L2 regularization before training of the network. This is also a separate optimization problem that needs to be solved. Bayesian deep learning uses Bayesian optimization techniques to find the network architecture and appropriate initial values of the learning parameters. In this thesis, basic convolutional deep learning and Bayesian deep learning are compared on a popular image classification problem. The performance, advantages and disadvantages of both methods are evaluated on the related classification problem. Although Bayesian deep learning significantly increased the classification performance on the test dataset, Bayesian optimization of the network structure and initial values of the learning parameters introduced an additional time cost.