Evaluation of human and robot faces and voices through behavioural and neuroimaging methods

Human-robot interactions have become important with the widespread use of robots in our daily lives. As we encounter robots from the simplest designs to humanoid designs, communication with robots becomes linearly easier. However, humanoid robot designs break this linear line and people may feel sudden revulsion and uncanny feelings towards these designs. This effect is called the uncanny valley effect. In our study; we investigated how the uncanny valley effect occurs, how voices and faces of robot and human designs are perceived by using behavioural and neuroimaging methods. 60 participants, 30 younger and 30 older, participated in the behavioural experiment. Unrealistic, semi-realistic and realistic voice and face designs were rated by the participants according to their feelings of normality and uncanniness. 35 healthy adults participated in the neuroimaging experiments, which is the second part of the thesis. Prefrontal cortex activities were recorded using fNIRS (functional near infrared spectroscopy) while the participants watched videos containing human and robot faces and voices. According to the results of the behavioural experiment step; multimodal auditory and visual stimuli elicited more uncanny feelings in participants than the unimodal auditory-only and visual-only stimuli. We indicated that the older participants showed more linear response from less realistic to realistic, but the younger participants showed a more sensitive response to different sound and image levels. In the results of the neuroimaging experiment; left lateral and partially right lateral areas of the prefrontal cortex were found to have a specific response to changes in robot and human voice and face stimuli. As a result of our study, it was determined that human and robot interactions are processed cognitively and physiologically in the human brain and create specific responses depending on different auditory and visual aspects. We also suggest that the uncanny valley effect has a physiological effect in the brain in addition to its cognitive effect. This exploratory study shows that human-robot interactions should be studied in a multidimensional manner with cognitive and neuroscientific methods. Keywords: Human-robot interactions, uncanny valley effect, face and voice design, prefrontal cortex, fNIRS measurement