Semantic expression extraction from images with depthwise separable convolution and LSTM networks

While it is easy for the human brain to accurately express the content of an image and what it tells us in sentences, it is very difficult for a computer. In order to create accurate and well-formed sentences, the language needs to be understood both syntactically and semantically. The biggest challenge is to create a description of not only the objects in the images, but also how they relate to each other and how they are related. With a deep learning approach, networks are trained on large datasets to understand objects, faces, scenes and other semantic information in images. Semantic analysis of images can be applied in many fields such as automotive, security, video surveillance and medical imaging. This field is constantly evolving and advancing with new deep learning models and large datasets that enable more accurate and complex analysis. In this study, feature extraction of 8000 images from the Flickr_8k dataset was performed with the Xception model. On the other hand, a unique lexicon structure was extracted from 5 descriptions of the images in Flickr_8k with LSTM. These two data were given to the transfer learning model to translate the images into natural sentences.