The effect of affective auditory stimuli on visual memory performance and decision making processes

The study aims to investigate the effects of different instructions on visual memory and decision-making processes when neutral human facial stimuli where emotion cannot be clearly determined and auditory stimuli containing three different emotional valences (negative, positive, and neutral) are presented simultaneously from different modalities. The sample of the study consisted of 65 (53 female, 12 male) university students aged between 18-27, who were determined in accordance with the inclusion and exclusion criteria of the study, and who agreed to participate voluntarily. Before the experimental phase of the study, due to the possibility of confounding variables, clinical, personality measurements and various scales related to emotion recognition and empathy level were applied to the participants, and a comprehensive neuropsychological test battery was applied to control the cognitive processes of the participants. During the experimental phase, neutral facial stimuli and emotional auditory stimuli are simultaneously presented to the participants with different instructions. When facial stimuli with neutral emotional valence and auditory stimuli with negative, positive, or neutral emotional valence are presented simultaneously, it is expected that the participants' emotional evaluation and decision-making performances and visual memory performances will be affected between different instruction groups (the group that received directive instruction / the group that did not receive instructional instruction) and depending on the emotional valence of the auditory stimuli. In this context, the experimental phase was created as 2 (instruction type; receiving directive instruction, not receiving directive instruction) x 3 (emotional valence of auditory stimulus; positive, negative, neutral) mixed factorial design. The participant groups in the study were randomly assigned to one of two groups according to the type of instruction, with 32 individuals (26 females, 6 males) receiving directive instructions and 33 persons (27 females, 6 males) not receiving directive instructions. Instruction type independent variables were formed as between-group design and auditory stimulus emotional valence independent variable as within-group design. Auditory-focused and visual-focused instructions are balanced and presented to the participants in the group that received the directive instruction. Participants in the directive instruction group are asked to evaluate their emotions by focusing on the auditory stimuli they hear in half of the presentation and by focusing on the neutral face stimuli they see on the screen in the other half of the presentation. For this reason, in order to examine the answers of the participants who received directive instruction in more detail, an additional 3 (instruction type; receiving visual-oriented instruction / receiving auditory-oriented instruction / not receiving directive instruction) x 3 (emotional valence of the auditory stimulus; positive, negative, neutral) mixed factorial design analysis was also carried out. Depending on the instructions given about the stimuli used in the study, the emotion evaluation and decision-making task regarding the target stimuli, visual memory task recognition performance and certainty evaluations, and the reaction times of all tasks were analyzed. In the study, mixed 2x3 and 3x3 ANOVA analyzes were used in order to examine the main effect of the instruction type, the main effect of the emotional valence of the auditory stimulus, and the interaction effect of these two independent variables. According to the results of the analysis obtained in the study, the main effect of the instruction type was not found to be significant in the 2x3 mixed factorial design, independent of the emotional valence of the auditory stimulus, whereas in the 3x3 mixed factorial design, it shows that different instructional manipulations have a significant basic effect on emotion evaluation and visual memory tasks and reaction times by affecting the attention of participants towards stimuli. While it was observed that the visual memory performances of the participants in the group who received the visual-oriented instruction were better in the direction of the instruction, the reaction times were found to be slower in the emotion evaluation performances. Results show that emotional evaluation and decision-making performances are affected in accordance with the emotional valence of the auditory stimulus, regardless of the instruction type. It has been determined that the emotional valence of the auditory stimulus has a context effect on the perception of neutral face stimuli. It was determined that the participants, regardless of the type of instruction they received, evaluated their emotions in a way that was compatible with the emotional content of the auditory stimulus they heard. It is seen that negative auditory stimuli accelerate the emotion evaluation reaction times for neutral face stimuli, creating a facilitating effect while slowing down the reaction times for decision-making responses and creating an inhibitory effect. It has been determined that negative auditory stimuli cause a decrease in recognition performance and certainty assessments in visual memory tasks and a longer reaction time for these tasks. When the interaction effects of the instruction type and emotional valence of the auditory stimulus were examined, it was observed that there were significant interaction effects in both mixed factorial design analyses, especially in the context of analyses negative auditory affect valence and instruction type. It was determined that the group that was not given a directive instruction benefited more from the emotional information from the other modality when the stimulus from one modality was unclear. The findings show that, regardless of the instruction type, the emotional valence of auditory stimuli affects the responses by creating an effective context on the emotion evaluation and decision-making task. Findings indicate that stimuli containing negative emotions have an advantageous effect on emotion evaluation reaction times, but have a disruptive effect on decision-making reaction times and visual memory performance. In the analyzes in which instruction manipulation was examined in detail, it was determined that participants' attention to stimuli was affected according to the type of instruction, and stimuli from different modalities in tasks affected their performance by changing the perception and attention process. In general, the study findings are important in terms of showing that when neutral face stimuli are given simultaneously with positive, negative, and neutral emotional stimuli presented from the auditory modality, it can affect the emotional evaluation and decision-making processes of the participants and this effect can also be reflected in the visual memory task performances performed immediately after the stimulus.