Real-time emotion recognition from EEG signals using one electrode device

Since the day the computer was first invented, there has been an interaction between people and computers. This interaction can be performed through software or hardware. The interfaces that provide interaction between people and computers are defined as Brain-Computer Interfaces (BCI). Electroencephalography (EEG) signals are preferred because of their practical use and portable structure in BCI applications. EEG technology measures the electrical activity of the brain. Many applications developed in recent years benefit from this activity. In recent years, researchers have concentrated on the development of ElectroEncephaloGraphy (EEG) based Brain-Computer Interfaces (BCI) to increase the quality of life using medical applications. In this context, BCI applications were introduced that could meet the basic needs of people with physical disabilities. In addition, the impact of the content produced in the marketing field on the users was tried to be determined with BCI applications. In recent times it has become an indispensable product for meditation practices and neurotherapy. Many games and entertainment applications were further enriched through the BCI. BCIs are used for marketing, gaming, and entertainment to provide users with a more personalized experience. Both medical and non-medical applications require the ability to interpret the user's multimedia-induced perception and emotional experience. The changes that both internal and environmental factors create in a person-specific manner are called emotions. Estimating the emotional state of a person is an important step to build effective BCI applications. Therefore, studies are needed to identify the correct emotional state. In the literature, there are few studies on real-time emotion recognition with EEG signals. With this study, it is aimed to close these deficiencies. It is also possible to foresee that neurotechnology will be the technology of the future. The day when people can upload their thoughts to computers is an unquestionable fact that these studies will prevail with applications that can increase the cognitive capacity of people. With the developments in EEG technology in recent years, EEG devices can be obtained with more reasonable costs. In addition, more complex advances are being made every day by using less channel EEG devices. The number of studies using a single-electrode device for emotion recognition is low, because single-channel EEG devices have recently been marketed. Single electrode devices are economical and easy to use. It is predicted that it will be included in the literature more widely in the coming years. In this thesis, a system is designed which detects instantaneous emotional state by using EEG data obtained in real-time with a single-channel commercial BCI device. While designing this system, it comes from the top of the following challenging problems: The collection of EEG signals, the selection of the most appropriate features, and the creation of the most appropriate detection model using these features. A software was written to collect the EEG signals. This software can report data coming in real time. After this software has been written, it has been tested whether the software works correctly by looking at the reported data. For the selection of the right features, firstly visual and auditory content was created to warn emotions correctly. For this reason, visual and auditory items that are both short in duration and as effective as possible have been chosen. The sufficiency of the content found was determined as a result of several pre-evaluation tests. After the content selection was completed, decision trees and random forest algorithms, which are widely used in the literature, were used to correlate EEG data with the emotional state of the content. The order of importance of the features is determined using the random forest algorithm. It has been decided which features should be in the final model by taking the order of importance of the features. In order to create the most appropriate model, the most suitable parameters were determined by using the deep learning algorithm and the grid search method for different parameters. For the suitability of the specified model, the data were divided into groups as training, validation and test data and the success rate was measured. It is also shown that the response time is fast enough to work in real time. The generated model is tested in real time and the response time is measured. It has been tested to produce accurate results under the required time constraints. The success of the system created on this issue has been revealed. Both the real-time and non-real-time classification success of the generated model is measured. In both cases, a success rate was achieved that would meet expectations. In addition, the system has been tested for both supervised and unsupervised learning using an incremental learning approach. All the parameters affecting the success of the system are discussed. The created model and method will be a pioneer for such studies. With the experimentation and the increase of collected data, even the ordinary events in human life will reveal the relation with the brain signals. To sum up, first, while the users watched a video consisting of film fragments and binaural beats that would trigger different emotional states, the EEG data from the users were collected. Then, pre-processing and feature selection were performed on this EEG data and an emotion recognition model with an accuracy of more than 87% was generated using deep learning neural networks. Using this model, a system which detects the instant emotional state on the queuing port structure of a certain length buffer using real time EEG data is designed. As a result, we present a real-time emotion recognition model using EEG signals. This thesis tackled to figure out the specific link between brain waves and the multimedia-induced emotions. Using an ANN-based classification scheme, the proposed system classifies the multimedia genres such as funny, horror, weepy with an average accuracy of 87% which cause emotional or psychological experiences that are induced in viewers. The system can be used for both medical and non-medical applications. In the future, we plan to develop a real-time game based on the fears of the players which will impact all events in the game. The game will come with a horror story which sets players up to solve a mystery while overcoming their fears. In the future, with the increase in work in this area, a society is hoped that people will not be sick and will be happy. But it is also necessary to work to ensure that these studies are not used outside ethical and moral boundaries. There is also the possibility that the ideological apparatus of the state may turn into a dystopic cluster by using such studies. For this reason, it is aimed to develop protocols in order to keep such applications in ethical limits in the future.