Music emotion recognition using convolutional long short term memory deep neural networks

In this thesis, we propose an approach for Turkish music emotion recognition based on convolutional long-short term memory deep neural network (CLDNN) architecture. For this purpose, a new Turkish emotional music database composed of 124 Turkish traditional music excerpts with a duration of 30 seconds each is constructed. We used novel features obtained by feeding convolutional neural network (CNN) layers with log-mel filterbank energies and mel frequency cepstral coefficients (MFCC) in addition to standard acoustic features. Classification results show that the best performance is obtained when the new feature set is combined with the standard features using the LSTM + DNN (LDNN) classifier. The overall accuracy of %99.19 is obtained using the proposed system with 10-fold cross-validation. When new features are added to the standard features, 6.45 and 5.65 points improvements are achieved for native listeners and experts, respectively. Additionally, the results also show that the LDNN classifier yields 1.61, 1.61, 2.42 and 3.23 points improvements for native listeners and 5.65, 0.81, 4.84, and 6.45 points improvements for experts in music emotion recognition accuracies compared to that of K nearest neighbors (k-NN), Sequential Minimal Optimization (SMO), Naïve Bayes and Random Forest (RF) classifiers, respectively.