Image captioning with convolutional and recurrent neural networks

Image captioning can be defined as describing a content of the image without human help. With the image captioning, it is aimed not only to understand the objects in the image, but also to understand the relations between the objects then turn it to the meaningful sentence. In this context, the problem of image captioning is located at the intersection of computer vision and natural language processing and it includes the difficulties of both fields. In order to solve the faced difficulties, different studies have been were put forward with different architectures. However, most of the studies in the literature have concentrated on languages belonging to the Germanic language family, especially in English. In the Turkish language, which is our native language, the number of studies in the image captioning field is very few. In this context, this dissertation conducted in the field of image captioning in Turkish language. Within the scope of the proposed study, models that are currently used to solve the problem of assigning titles to images in English language and new models which created with some improvements made on the previous models have been trained with the dataset processed by considering the agglutinative structure of the Turkish language. In this study, some models have been created by ensemble learning and some models have been created by attention mechanism. All models that are created have encoder-decoder architecture and these models consist of convolutional and recurrent neural networks. The outputs of the models were put into sentences by greedy search or beam search techniques. Finally, created sentences were evaluated with the BLEU score method.As a result of the experiments, the highest BLEU-1 score of 0.42 was obtained. The score obtained was the highest score among the Turkish dataset studies in the field of image captioning.