Grading of knee osteoarthritis based on the integration of deep and traditional image features via early and joint fusion strategies

Knee Osteoarthritis (KOA) is a chronic joint disease commonly observed in the elderly population and significantly reduces quality of life. Since conventional diagnostic methods often rely on expert interpretation, they are prone to subjective errors and may lead to delays in treatment. Therefore, developing automated diagnostic systems that can provide more reliable and objective decisions is of great importance. In particular, determining not only the presence of KOA but also its severity accurately is a critical requirement for ensuring timely and effective clinical interventions. In this thesis, a multi-component approach is proposed to improve the accuracy and reliability of KOA grading by integrating handcrafted and deep image features through early and joint fusion strategies. Handcrafted features such as morphological, statistical, GLCM, HOG, LBP, wavelet, perimeter, area, and Zernike moments were extracted from X-ray images and combined with deep representations obtained from ResNet-50, VGG16, EfficientNetB0, Xception, and DenseNet201 models. These features were directly fused in the early fusion strategy, while in joint fusion strategies, representations from different levels were integrated. Accordingly, KOA grades were categorized into five classes "normal," "doubtful," "mild," "moderate," and "severe" based on the Kellgren-Lawrence (KL) grading system. The combined feature representations were evaluated using Artificial Neural Network (ANN)-based classification architectures. Additionally, data augmentation and the Synthetic Minority Over-sampling Technique (SMOTE) were applied to mitigate class imbalance and ensure balanced representation of each KOA grade. The proposed fusion strategies enriched information representation and enhanced inter-class separability in KOA grading. The highest accuracy and F1-score values obtained for each fusion strategy were recorded as follows: 73.56% and 71.46% for Early Fusion Type II, 83.50% and 83.15% for Joint Fusion Type I, and 85.39% and 84.96% for Joint Fusion Type II, respectively. In the most successful configuration, Joint Fusion Type II, both handcrafted features related to tissue and morphology and deep representations derived from the VGG16 model played a significant role in distinguishing KOA grades. This multi-level feature integration enabled the reliable identification of both early and advanced KOA stages, providing a robust AI-based framework suitable for clinical use. Keywords: Knee Osteoarthritis Grading, Traditional Image Features, Deep Learning-Based Features, Feature Fusion Strategies, Artificial Neural Network Classification