Polyp segmentation and classification of gastrointestinal findings from endoscopic images: The effectiveness of transformers and hybrid models

Gastrointestinal (GI) system diseases are common worldwide and cause both death and a decline in quality of life. Therefore, early and accurate diagnosis is vital for successful treatment; timely detection of findings such as polyps, ulcers, and tumors plays a critical role in preventing disease progression. While endoscopic images serve as valuable diagnostic tools, their manual evaluation is time-consuming and prone to errors. For this reason, AI-supported automatic analysis methods are becoming increasingly important to assist clinical decision-making processes. This thesis investigates the effectiveness of Transformer-based and hybrid deep learning architectures in polyp segmentation and GI finding classification using the Kvasir v2 dataset. Vision Transformer (ViT), Swin Transformer, ConvNeXt, and ConViT were used for classification tasks, while SegFormer and Swin Transformer + UPerNet were employed for segmentation. After comprehensive preprocessing, training was conducted using both random data splitting and 5-fold cross-validation strategies, and the models were evaluated using multiple metrics, including accuracy, F1-score, Dice, and IoU. In classification, the ConvNeXt model achieved the highest performance with 98.50% accuracy and 98.46% F1-score under random splitting, and 98.13% accuracy and 98.25% F1-score with cross-validation. For segmentation, the Swin Transformer + UPerNet model stood out with 97.24% accuracy, a Dice score of 0.9025, and an IoU of 0.8303, while SegFormer reached 96.20% accuracy and a Dice score of 0.8731. Transformer architectures provide notable advantages in analyzing complex structures in medical images due to their ability to model long-range contextual relationships. Hybrid models further enhance generalization by incorporating convolutional inductive biases. The findings demonstrate the applicability of Transformer-based approaches in medical image analysis and contribute to the development of clinical decision support systems.