Efficient image annotation and caption system using deep convolutional neural networks

In recent years, with the advances in the artificial intelligence field, image annotation also known as image description (IAC) has progressively attracted researchers' attention. IAC automatically creates natural text descriptions according to the image contents. IAC combines the knowledge of computer vision and natural language processing. In this research, a novel image annotation and description system was developed. The main parts of the developed system are Convolution Neural Network (CNN) and Long Short Time Memory (LSTM). Also, the developed system was enhanced by multiple steps such as adding regularizing to convolution layers, adding dropout layers to the fully connected layers, using genetic algorithms to find the most suitable batch size, and investigating the performance of multiple optimizers such as Adaptive Moment Estimation (Adam), Stochastic Gradient Descent(SGD), and Nesterov accelerated gradient to find the most suitable one for the developed approach. The developed system was validated by multiple experiments using one of the challenging datasets, i.e., the Flicker dataset. Overall, our improved model outperformed the existing state of arts using the BLEU metric. Also, results prove that the designed system can effectively describe images. Last but not least, this research help researchers by highlighting some open challenges in the field of image annotation.