Investigation of usability of artificial intelligence semantic video processing methods in medicine

Aim: Object scanning or recognition is a general term that describes a collection of related computer vision processes that involve identifying objects in digital images. Recently, studies on the use of object recognition applications in the field of health as clinical decision support systems are gaining importance. Thanks to the developing computer and machine learning technologies, detection of disease or anomaly via clinical images and videos (computed tomography, ultrasound, etc.) has been made automatically by computers. To perform these operations having high calculation cost with high accuracy and precision, purposeful deep learning architectures have been developed. Convolutional neural networks (CNN), one of the frequently used architectures for object recognition processes, is a deep, feed forward neural network class. In this study, by using appropriate video/image processing techniques and CNN architecture, it is aimed to develop a user-friendly software for healthcare professionals with various methods such as detection, identification, classification and tracking of polyps contained in the endoscopic images. Material and Methods: The dataset consisted of 300 images in total. These images are images described and validated by medical doctors (experienced endoscopists) of several classes, consisting of hundreds of images for each class, such as anatomical milestones, pathological findings, or gastrointestinal procedures in the digestive tract. The images were obtained from the web address https://datasets.simula.no/kvasir, which is open source for research and educational purposes. CNN and Max-Margin object detection method (MMOD), one of the deep neural network architectures in the Dlib library, was used in the modeling phase. The simple cross-validation (i.e., hold-out) method is used and the whole data is divided into two parts: 80% for training and 20% testing. In the evaluation of model performance, precision, recall, F1-score, average precision (AP), mean average precision (mAP), optimal localization recall precision (oLRP), mean optimal LRP, (moLRP) and intersection over union (IoU) were used. Results: In the implementation of the study, when the previously described steps on the open access video image dataset related to the colonic polyps were performed, all performance metrics examined in the training dataset were 100%, while precision of 98%, recall of 94%, F1-score of 94%, AP of 89% and mAP of 89%, oLRP of 48% and moLRP of %48 were calculated on the testing dataset. Conclusion: Considering the values of the calculated performance criteria, it was found that the proposed system gave successful predictions in the diagnosis of gastrointestinal polyps. Keywords: Object recognition, deep learning, decision support system, gastrointestinal polyps, convolutional neural networks