Radiological image and text based medical concept detection in social networks with deep learning

Today, it is seen that social networks on health topic are increasing day by day. With the effect of these networks, a large number of medical images emerge that are identified and interpreted by various experts. Therefore, concept detection and image classification from medical images remains a challenging task. In recent years, studies in this field have increased the activities on deep learning as a model. The most important reason why the interest in the deep learning model has increased so much is that there is enough data to be trained and the necessary physical infrastructure is ready to process this data. The main purpose of the thesis work is to perform multi-label classification of images by automatically selecting the medical concepts that should be assigned to the radiological images shared on a social network. Concepts come from the Unified Medical Language System (UMLS). In order to predict the concepts in the study, convolutional neural network (CNN) combined with feed forward neural networks, and various image encoders (VGG-19, ResNet-101, DenseNeT-121, Xception, Efficient-B7) are employed. The proposed hybrid deep learning models have been tried and tested on the ImageCLEF 2019 dataset. Then, the performance of the models was evaluated on the data set (Rdpd_Test_Vs) formed from the radiology images and their comments collected over the social network. Evaluation is performed in terms of F1 scores between system prediction and absolute accuracy concepts. Evaluation results are promising and have high performance.