A new approach to disease recognition using zero-shot learning and ontology

Although open image data sets have increased recently, data sets are still limited. So disease detection/recognition with these data sets and labels is very costly and hard challenge in medical domain. Therefore, researches on this problem still need to be improved. It is a greatest challenge to diversify data sets and has to face the problem of subjectivity. Classes without instances, in other words unseen classes, can be trained with Zero-Shot Learning (ZSL) in order to overcome this problem. In this thesis, it is aimed to strengthen ZSL by using ontology as an auxiliary information for class embeddings. In addition to the procedures used in existing ZSL methods where image embeddings and class embeddings are mapped, the semantic data of class embeddings are also considered. In the proposed approach, ZSL is powered by image embeddings and class embeddings of the multi-label ChestX-ray14 data set, as well as semantic data from DBpedia. Cosine, Hamming and Euclidean distances are considered to maximize similarities. Classical and convolutional neural networks were used to augment the data set and the precision value was tried to be increased by making tests with each of them. The ResNet50 neural network is trained on the multi-label ChestX-ray14 data set with different parameters. The highest precision value of 23.25% was obtained in one-to-one matching and 45.73% precision value was obtained in at least one matching. With these results, which have a high success rate compared to the success rates in the studies in the zero-shot learning literature, it is thought that this thesis will make an important contribution to the field of medical imaging by detecting/recognizing invisible disease.