Deep learning based voice emotion analysis in intercom systems

Push-to-Talk systems in subways are systems that allow passengers to communicate with the engineer. Through this system, passengers can contact the engineer in case of emergency and tell him/her that there is a problem. However, since there is no priority in the current Push to Talk/Intercom systems, the engineer can talk to someone who presses the button randomly. Therefore, the passenger who presses the button for an important/emergency situation may be delayed to be interviewed later and may cause late intervention. In this study, Wav2Vec2, one of the recently developed deep learning architectures, is used to solve these problems in push-to-talk systems. When the passenger presses the button on the push-to-talk device, the human voice recognition system is activated. When the passenger presses the button on the push-to-talk device, the trained model is used to classify the emotion and detect whether the passenger is really in an emergency or not. The aim of this study is to integrate the trained emotion classification model into the system and to develop system automation according to the predicted emotion class, to pass non-emergency situations with an automatic system response and not to delay emergency situations. In the study, audio data in the EMO-DB dataset were converted into 2D images, i.e. audio spectrograms, by transfer learning using a Wav2Vec2 model pre-trained according to emotion classes. It was observed that the model trained by transfer learning with the obtained data set achieved 92% accuracy.