Breast cancer detection and classification with deep learning approaches

Breast cancer is one of the leading causes of cancer-related deaths among women worldwide. Early detection and accurate diagnosis are crucial for improving survival rates and planning effective treatment. Mammography is the primary imaging method for breast cancer screening, detecting abnormalities at an early stage. Among mammographic findings, masses are the most common and diagnostically significant lesions. Mammograms are often evaluated manually by radiologists; however, the diversity of lesions, dense breast tissue, and large image volumes make this process challenging, time-consuming, and labor-intensive. Furthermore, differences in experience levels and attention fatigue caused by prolonged examinations can negatively affect diagnostic accuracy. In recent years, artificial intelligence applications, particularly those based on deep learning, have made significant contributions to medical imaging diagnostics. The You Only Look Once (YOLO) algorithm, a leading approach in this field, stands out with its real-time object detection and classification capabilities. This allows for the rapid and accurate identification of abnormalities in mammograms, offering potential support to clinical diagnostic workflows. In this thesis, a YOLO-based deep learning model was developed to detect and classify breast masses in mammograms. The YOLOv5, YOLOv8, and YOLOv9 (GELAN) architectures were utilized, and various attention modules and convolutional blocks were integrated into these models to generate different model variants. These models were trained from scratch using CBIS-DDSM, VinDr-Mammo, and their combined dataset. Then, fine-tuning was performed on the INBreast dataset using five-fold cross-validation. Furthermore, the effects of different data augmentation strategies on model performance were analyzed. The best-performing model variants on the INBreast dataset were further fine-tuned using a dataset obtained from Başkent University Ankara Hospital to evaluate clinical applicability. In the CBIS-DDSM + VinDr-Mammo -> INBreast scenario, the model variant (GELAN-c + DWConv + CBAM), which integrates depth-wise convolution (DWConv) and the Convolutional Block Attention Module (CBAM) into the GELAN-c architecture, achieved a mAP@0.5 score of 0.878 with the default data augmentation configuration, representing the highest performance in the study. In the CBIS-DDSM + VinDr-Mammo -> Başkent scenario, the model variant (YOLOv5s + EMA) created by integrating the Efficient Multi-Scale Attention (EMA) module into the YOLOv5s architecture, achieved a mAP@0.5 score of 0.848 using the default + mixup augmentation strategy. These results demonstrate that the proposed YOLO-based model variants achieved high performance on both public datasets and real clinical images and show strong potential for integration into clinical decision support systems.