Robust keyword spotting in noisy environments based on deep learning

The present thesis introduces a novel keyword spotting (KWS) system aimed at enhancing performance under adverse noisy conditions by integrating supplementary acoustic information through a transformer-based meta-classifier framework. To accomplish this, KWT-1—a variant of the Keyword Transformer family introduced by Berg et al.—is reimplemented as the base KWS component. This model is applied without the use of knowledge distillation and is trained on the Google Speech Commands v2 dataset for a 12-label classification task. Extensive data augmentation strategies are employed in alignment with the original study to ensure robust model performance. To extract complementary acoustic features, two additional modules are integrated: a noise type classifier and a signal-to-noise ratio (SNR) prediction model. The noise classifier is implemented as a one-dimensional convolutional neural network informed by the methodology of Abdoli et al. and trained on the UrbanSound8K dataset to recognize ten distinct environmental noise classes. The SNR prediction model adopts a novel hourglass-style convolutional architecture to perform continuous SNR regression. During its training, clean speech samples from the Google Speech Commands v2 dataset are mixed with noise from UrbanSound8K at random SNR levels ranging from 0 to 20 dB, simulating realistic acoustic environments. The outputs from the three branches—keyword prediction, noise type, and estimated SNR—are fused at the decision level using a transformer-based meta-classifier. In this configuration, each model output is treated as a discrete token, projected into a shared embedding space, and processed by a transformer encoder block. This design is intended to capture the complex interdependencies between semantic, environmental, and acoustic factors. Although the proposed fusion model did not surpass the performance of the standalone KWT-1 baseline in terms of keyword classification accuracy, the work contributes to the academic literature by introducing an hourglass-style CNN for SNR level estimation that outperforms existing neural network-based approaches. Evaluation is conducted using classification accuracy for keyword and noise type detection tasks and mean absolute error (MAE) for the SNR regression task.