Deep learning-based spectro-temporal and cnn feature fusion for deepfake speech detection and synthesizer recognition

Recent advancements in text-to-speech (TTS) and voice conversion (VC) technologies increasingly threaten the authenticity of audio content, thereby intensifying the need for effective detection methods. This study proposes a novel two-stage detection framework that integrates deep Convolutional Neural Network (CNN) representations extracted from Mel-spectrograms with various temporal features. In the classification stage, an ensemble model composed of Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM) algorithms is employed, and their outputs are combined using a majority voting strategy. A custom dataset was collected from a total of 135 speakers, including mono and stereo recordings at sampling rates of 44.1 kHz and 48 kHz. In addition, synthetic variants of each original recording were generated using CoquiTTS, DiffVC, and FreeVC systems. Comprehensive ablation experiments demonstrated that using only CNN-based features achieved an accuracy of 9.3%, whereas incorporating temporal features increased the accuracy to 100%. In five-fold cross-validation experiments, the EfficientNet-B0+ model with feature fusion achieved accuracy values ranging from 92% to 98% across the most challenging folds. The generalizability of the proposed method was validated across multiple CNN architectures, including EfficientNet-B1, EfficientNet-B4, EfficientNet-V2-S, EfficientNet-V2-M, Xception, and ResNet-50. The results indicate that performance is largely driven by feature engineering rather than model architecture. Grad-CAM visualizations further confirmed that the model focuses on subtle yet critical phonetic cues that distinguish real from synthetic speech. Overall, the findings demonstrate that combining deep learning features with handcrafted temporal representations within an ensemble framework provides a robust and transferable defense against emerging TTS- and VC-based audio spoofing threats.