Multi-teacher based knowledge distillation for retinal vessel segmentation

Convolutional neural network (CNN) has achieved significant success in recent years for tasks such as vessel segmentation in medical imaging. However, accurately segmenting thin vessels in retinal images remains a challenge for existing methods. Therefore, new approaches are needed to develop more effective and generalizable models. In this study, a novel framework called Multi-Teacher Based Knowledge Distillation (MTKD) is proposed. The proposed method leverages the outputs of teacher models specialized in different vessel types (thin, thick, and general) and transfers this knowledge to a single student model. The developed method was evaluated on four different retinal datasets: DRIVE, CHASEDB1, CHUAC, and DCA1. Results show that the proposed MTKD approach outperforms classical U-Net and advanced methods, particularly in the segmentation of thin vessels. On some datasets, it achieved up to a 2.8% improvement in F1-score compared to state-of-the-art methods. Moreover, the method maintained high levels of accuracy and specificity while demonstrating strong generalization capability. In addition, comprehensive ablation studies were conducted to analyse the effects of different components of the proposed framework. These studies assessed various knowledge distillation strategies, loss functions, temperature settings, and teacher contributions, and appropriate hyperparameters were determined to enable more effective learning for the student model. The findings revealed that the penalty-based loss function played a significant role in improving model performance. While the MTKD approach offers a high-performing and reproducible solution, it was observed that its sensitivity may occasionally fall behind some advanced methods. Future studies may explore feature-based knowledge distillation strategies that transfer intermediate layer representations from teacher models to further enhance segmentation performance. Furthermore, adaptive loss functions that respond to data complexity and architectural optimization techniques may improve both accuracy and computational efficiency.