Application of novel filter banks and fundamental frequency detection method in speech emotion recognition with deep learning

In this manuscript, a novel filter bank design, named EFB and a pitch determination algorithm, HDM, are proposed. The proposed filter banks are aimed to replace current state-of-the-art MFCC and Mel filter banks. We hope that EFB filters will have great impact over the speech emotion recognition applications. Today, most of the speech processing applications use Mel filters or its transformed and reduced version MFCC. There are various other filter banks proposed to imitate the human ear structure. However, these models have too many redundant frequency regions. MFCC contains fewer coefficients but computation of DCT is a setback of speed. Another disadvantage of these filters is the difficulty to interpret the MFCC values. It is very hard to gain an insight by inspecting the Mel filters or MFCC. The EFB filter banks are not only fast and easy to compute compared to the Mel and MFCC, but it also provides more insights about the underlying structure of the speech waveform. In this study, EFB filter bank is implemented on emotional speech datasets (EmoSTAR, EmoDB, IEMOCAP, MELD) with various Deep Learning Architectures and SVM-SMO classifier to compare them with MFCC and Mel filter banks. We also investigate feature selection and data augmentation methods. Prosodic features are used very extensively in speech emotion applications. For this part, we developed a novel fundamental frequency calculation method called HDM which exploits the intervals between the harmonics of vowel speech sounds. We test the HDM against some of the prominent algorithms such as autocorrelation, CREPE, YIN, YAAPT, cepstrum, and FCN on Hillenbrand Vowel dataset, Texas Vowel dataset, and vowel part of TIMIT dataset for narrowband telephony speech as well as wideband speech.