Theses supervised by Dr. Öğr. Üyesi Burcu Ayşen Ürgen
10 theses · İhsan Doğramacı Bilkent University
Yukarıdan aşağıya dikkat süreçlerinin eylem algısı ağı üzerinde etkisi
Action perception is one of the fundamental skills for survival and social interaction. In neuroscience literature, it is well-established that when an action is visually perceived, the Action Observation Network (AON) becomes active in the human brain. This network has three core brain regions: the posterior superior temporal sulcus (pSTS), parietal cortex, and premotor cortex. Recent studies in neuroscience have shown that action perception is not a passive process but is affected by top-down signals such as attention. In this study, we investigated the influence of attention on the AON by conducting a two-session fMRI experiment. The stimuli comprised eight videos of pushing actions, with each video featuring a unique combination of actor (female or male), effector (hand or foot), and target (object or human). In the active fMRI session, participants viewed the videos while performing three different tasks that directed their attention to distinct features of the action (actor, effector, target). In the passive fMRI session, participants viewed the same videos without performing any task. The data from the passive session was used to extract ROIs: pSTS, parietal, and premotor in each hemisphere. Univariate analysis, representational similarity analysis (RSA), and decoding analysis were performed. Univariate analysis showed that introducing attentional demands during video viewing elicited a significant increase in neural activity within both parietal and premotor cortices relative to passive viewing conditions. RSA results revealed significant correlations between neural activity patterns and task models across all ROIs, indicating top-down influence throughout the AON. Decoding analysis showed unique top-down effects in each ROI, depending on its hierarchical level and intrinsic selectivity. These findings demonstrate strong top-down modulation of the AON based on cognitive demands, highlighting the dynamic interplay of attention and action perception.
Dikkatimizi ne çekiyor? Robot ve insan tarafından yapılan iletişim içeren ve içermeyen hareketlerin farklı algısal yük koşullarında görsel dikkat incelemesi
This study investigated how agent type (human vs. robot), communicativeness (communicative vs. non-communicative), and perceptual load (low vs. high) interactively influence visual attention and task performance. A 2 (Agent) × 2 (Communicativeness) × 2 (Perceptual Load) within-subjects design was employed with 34 participants. To examine attentional allocation under perceptual load, Areas of Interest (AOIs) were defined for the central letter array and the peripheral agent videos. Behavioral performance (reaction time, accuracy) and multiple eye-tracking metrics (total dwell time, dwell time per fixation, first fixation latency, saccadic velocity, pupil dilation) were analyzed. Behavioral results confirmed the effect of the perceptual load manipulation, with participants being significantly less accurate and slower in high-loaded conditions. In the presence of a distractor, reaction times and accuracy were further modulated by significant interactions between agent, perceptual load, and communicativeness. Eye-tracking analyses revealed that initial orienting was driven by the communicativeness and perceptual load, when communicative cues captured attention faster only under high load. Dwell time per fixation remained stable across the conditions. On the other hand, total dwell time revealed a critical three-way interaction. Under high load, communicative actions performed by humans increase total dwell time, whereas if the action is performed by a robot, dwell time decreases. Pupil data supported this, indicating that under high load, non-communicative actions required more cognitive effort than the communicative ones. Initial orienting of eye metrics was guided by task demand; overall engagement with the nature of the action is a product of a complex and sensitive cognitive process sensitive to perceptual load. Robotic social cues are processed dissimilar compared to human ones.
Dikkat yükü altında biyolojik hareket algısının nöral temelleri
Humans can detect and differentiate biological motion from non-biological motion stimuli effortlessly, even if the stimuli were shown as simplistic as a composition of moving dots (i.e. point-light displays [PLD]). Considering its survival and social significance, BM perception is assumed to occur automatically. Indeed, Thornton and Vuong (2004) showed that task-irrelevant BM in the periphery interfered with task performance at the fovea. However, the neural underpinnings of this bottom-up processing of BM lacks thorough examination in the field. Under selective attention, BM perception is supported by a network of regions including the occipito-temporal, parietal, and premotor cortices. A retinotopy mapping study on BM showed distinct maps for its processing under and away from selective attention (Saygın & Sereno, 2008). Based on these findings, we investigated how bottom-up perception of BM would be processed under attentional load when it was shown away from the focus of attention as a task-irrelevant stimulus. Participants (N=31) underwent an fMRI study in which they performed an attentionally demanding visual detection task at the fovea while intact or scrambled PLDs of BM were shown at the periphery. Our results showed the main effect of attentional load in fronto-parietal regions; as well as, the main effect of peripheral stimuli in occipito-temporal cortex. Both univariate and multivariate pattern analysis results support the attentional load modulation on BM. Lastly, ROI results on each core node of BM processing network expanded these findings by showing that the attentional load modulation on both intact and scrambled BM stimuli were the strongest in bilateral occipito-temporal regions as compared to parietal and premotor cortices. In conclusion, BM was processed within the motion sensitive regions in the occipito-temporal cortex when shown away from the selective attention, and was modulated by attentional load.
Otonomi ve robotonomi: Toplumsal cinsiyetin rolü
This study examines how gender affects people's perception of autonomy in social robots. It investigates whether an agent's name (masculine/feminine/neutral/ technical/no-name) and type (human/robot) can impact agency, personification, competency, and gender evaluations and whether these evaluations are at an unconscious level. The study consists of a pre-study and a main study with 150 participants. Participants watched 4-second 18 videos in the main study and answered autonomy-related questions. They also completed three implicit association tasks, adapted according to the study's themes, autonomy, agents, and gender. ANOVA analysis was conducted to analyze the agent, naming, and gender effects for the explicit part. The d-score was calculated for all participants in the implicit part, and an ANOVA analysis was conducted. Regression analysis was conducted to determine gender attribution in the pre-study and main studies. Correlation analyses were also conducted to determine if explicit-implicit parts were correlated. Lastly, a thematic analysis was conducted for qualitative inputs in the explicit part and categorized into nine themes. The study found the main effect of action and agent in agency-level attribution. In competency and gender attribution, agents had no main effect in none of the name conditions. In the implicit part, women and men participants differed in men-independent/women-dependent association IAT-1. The other two IATs, women and men participants, responded similarly. The study suggests that name manipulation does not affect people's autonomy perception, but rather agent types and actions characteristics affect them. Furthermore, people's implicit and explicit answers do not predict each other.
Çift dilli bakış açısıyla duygusal prozodi ve aksan
People use different kinds of cues to process and understand spoken language in their daily lives. Understanding language is not solely based on the processing of grammar and words but also the processing of paralinguistic (non-verbal aspects of spoken language) cues, which have an important role in the ultimate comprehension of spoken language. Emotional prosody is one of the crucial paralinguistic cues and is the focus of the present thesis. It refers to verbal cues that convey different types of emotions based on changing pitch, timbre and tone of voice during speech. Different theories attempt to explain how emotional prosody and its related cues are cognitively processed during speech comprehension, such as the Universality Hypothesis, the In-group Advantage hypothesis and Dialect Theory. This thesis examined the processing of two different paralinguistic cues (emotional prosody and accent) and were discussed within the perspective of these theories. This study had 3 main research questions: (1) how do bilingual individuals differently process emotional prosody and different accents in their first (L1) and second (L2) languages? (2) How do different accents affect the emotional comprehension of L1 and L2 speakers? and (3) is the processing of vocal emotions universal or does it include culturally-specific aspects? Bilingual Turkish-English participants were tasked with listening to sentences in either their L1 (Turkish) or L2 (English) that varied in their emotional prosody (happy, sad, neutral tones of voice) and their accent (accented Turkish; accented English). Electroencephalography (EEG) recordings were taken to measure the P200 event-related potential response, which indexes automatic emotional processing and has been shown to vary between first and second languages. The results showed that Turkish-English bilingual individuals showed higher P200 amplitudes when they heard emotionally vocalized sentences in their L1 compared to sentences vocalized with neutral tone of voice, regardless of the accent. However, in their L2, they showed higher P200 amplitudes for neutral sentences compared to emotionally vocalized sentences. Overall, these results propose that the nativelikeness of the stimuli is crucial to emotional prosody processing (in line with the In-Group Advantage hypothesis), such that we process emotional information more automatically in our L1, regardless of the speaker.
Aktör algılama sürecinde görsel belirsizliği çözmede ön bilgi ve hareket ipuçlarının etkileşimi
Agent perception, a complex cognitive task that involves interpreting an agent's actions to infer their internal states and adjust our behavior accordingly, is a fundamental aspect of human cognition. Despite its importance, a significant gap persists in our understanding of how low-level factors, such as motion and form information, interact with top-down factors, such as prior information about the agents. To address this gap, we conducted two electroencephalogram (EEG) experiments investigating the interplay of prior knowledge and motion information on the temporal dynamics of agent perception in the human brain. We used human, android and robot agents engaged in various actions. The chosen agents were designed to form a set wherein form information alone could not easily resolve the ambiguity in agent identities. In the first experiment, participants were informed about agent identities before the experiment (Prior Experiment), while in the second experiment, participants remained uninformed (Naive Experiment). We controlled the availability of motion information by presenting stimuli in either video and image formats. We recorded scalp EEG and utilized event-related potential (ERP) analysis and model-based representational similarity analysis (RSA) to uncover the temporal course of agent representation in the human brain. Our results revealed that the processing of agents in EEG depends on the availability of motion information and prior information. Specifically, in the Naive Experiment, agent information was available longer during the still condition than in the moving condition. In contrast, agent information was present for similar durations in still and moving conditions of the Prior Experiment. These findings suggest that prior knowledge and motion information interactively modulate the duration of the processing of agent information. Our results underscore the critical role of prior knowledge and motion cues in shaping the processing of agent information, highlighting the complex interplay between top-down modulation and bottom-up cues in driving the perception of agents and their actions. This study contributes to our understanding of the temporal dynamics of agent perception and the role of top-down and bottom-up processes in this complex cognitive task.
Sosyal robotlarda açık ve örtük ölçümlü zihin algısının bireysel farklılıklar modülasyonu
The attribution of mental states to the object or subject that an individual interacts with according to its appearance or behavior is called mind perception (Gray et al., 2007). Recent research on human-robot interaction has shown that robots can create mind perceptions like other agents under certain conditions. In addition, while the two dimensions of mind perception (Agency and Experience) are mostly controlled using explicit measurement methods in the literature, the use of implicit measurement methods in the measurement of mind perception is still almost nonexistent. In addition to this fundamental gap, studies examining mind perception in robots have investigated how appearance affects mind perception, while the effect of action perception almost again has never been observed. In this context, we investigated how robots affect mind perception by manipulating differences in action and appearance. Methodologically, we conducted our study using both the explicit measurement method and the implicit measurement method due to the gap in the literature. In this study, individual difference measurement was also used to observe the causes of different attributions in mind perception to robots. In the first study, participants (N=102) evaluated how the robots' performing different actions (biological, verbal and nonverbal communicative and neutral) and appearance (humanoid and mechanical) affect mind perception; in the second study, participants (N=185) evaluated the effect of robots' actions and appearances on mind perception in terms of implicit and explicit measurement methods. In addition, 11 individual difference measures were used to observe individual differences that modulate mind perception. Looking at the results, it has been observed in both studies that the action of robots affects mind perception. In the explicit measurement method, neutral behavior was found to create less mind perception than communicative and biological action. In the implicit measurement method, differences in reaction time were observed between communicative actions and biological & neutral actions. Individual differences that modulate the perception of the explicit and implicit mind have been observed. According to this, intentionality of behavior, theory of mind, and perception of loneliness are core modulates for explicit mind perception, while negative mood primarily modulates implicit mind perception. Looking at the results, it was observed that the perception of action had an effect on the mind perception, the implicit and the explicit mind perception showed different patterns from each other, and the individual differences predicted the pattern of implicit and explicit mind perception.
Ön bilginin biyolojik hareket algısı üzerindeki etkilerinin davranışsal ve nöral olarak incelenmesi
The capacity to understand the actions of others, a cognitive phenomenon known as biological motion perception, is crucial for humans. Recent research demonstrates that biological motion is processed distinctively compared to the motions of inanimate objects. A dedicated brain network for processing biological motion and actions has been uncovered through fMRI studies. M/EEG studies have revealed time windows within which biological motion processing occurs. Despite these findings, a comprehensive understanding of the fundamental mechanisms driving biological motion perception, especially the effects of top-down processes, and the temporal dimension of these effects still remain unexplored. Recent evidence in visual perception suggests that prior knowledge and expectations affect visual perception; however, the generalizability of these effects to socially important stimuli, such as biological motion, is still unknown. This study aims to illuminate the effects of prior information on the behavioral and neural mechanisms of biological motion perception. To this end, we conducted a series of behavioral experiments and an EEG experiment to investigate the effects of prior information on biological motion perception. Through our behavioral experiments, we found that prior information influences the individuation process of biological motion, albeit conditionally. Specifically, this influence is observed only when the cue carries information about the type of action in the biological motion stimuli, and the reliability of the cue is high, at 75%. Our EEG experiment demonstrated that correct and incorrect prior information affects the temporal dimension of biological motion perception, suggesting an early effect of prior information during biological motion processing. Moreover, a comparison of the temporal generalization matrices suggested that correct prior information accelerates biological motion perception by accelerating the formation of related representations in the brain relative to the neutral condition. Additionally, the temporal generalization analysis results illustrate a sequence in representations within brain activity: the representation of location information precedes the representation of action type of biological motion. These results suggest that computational models, developed to model the underlying mechanisms of biological motion perception, should consider the implications of predictive processes and their temporal dimension. Furthermore, these findings support the applicability of predictive models to not only low-level stimuli but also to higher-level stimuli.
Biyolojik Hareketin Dikkat Yükü Altında Yukarıdan Aşağı İşlenmesindeki Kortikal Bölgelerin Etkin Bağlanırlığı: Bir iMRG-DNM Çalışması
The ability to detect biological motion holds an evolutionarily important role in vital and social functions. However, in our daily lives, we perceive biological motion while we are at a task most of the time. In other words, it is perceived when our attention is directed at another thing. In this aspect, understanding the dynamics of its bottom-up perception is of high importance. Meanwhile, the attentional mechanisms and where their effects occur are a matter of debate in the literature, sparking off various theories, such as early selection, late selection, and attentional load theory. Dynamic causal modeling (DCM) is a suitable tool for investigating the dynamics of attentional effects on the network, enabling the bottom-up perception of biological motion, and comparing the existing theories in the literature with Bayesian graph models. To this end, we utilized the DCM approach with fMRI data collected using an attentional load paradigm and biological motion peripheral distractors. In our model space, we modeled the theories of selective attention along with two complementary models. The Bayesian Model Selection (BMS) showed that the model that explained the data the best was the model where both attentional load conditions modulated all top-down connections rather than the models of existing theories. This showed that attentional effects take part in the bottom-up perception, not in a focused location, such as early or late, but in a more distributed manner throughout the processing pipeline. Further statistical tests on the model parameters yielded no difference between load conditions and between biological motion and scrambled motion in their modulation strengths. Yet, the strengths of biological motion on different connections were different from each other. A similar observation is also made for the low load condition but not for the scrambled motion and high load conditions. The former can be accepted as evidence for the differential processing of biological and scrambled motion. The latter may be explained by a spillover of perceptual resources on biological motion and causing competition in low-load conditions.
Doğal eylem algısının nöral temellerinin algısal karar verme teorik çerçevesinde incelenmesi
Every day, we perceive many actions performed by people around us. Understanding the movements we perceive and the intentions behind these movements is critical to determining how to respond to the observed action. Neurophysiology studies and computational model theories suggest that this perceptual decision process consists of two stages: accumulation of encoded sensory evidence and selection of response. EEG studies in humans have identified a neural trace thought to represent the first of these two stages: Centro Parietal Positivity (CPP), which scales with the strength of the incoming sensory evidence and differs from the Lateralized Readiness Potential (LRP) that represents motor preparatory activity. However, these studies mainly utilized simple and artificial stimuli; therefore, whether perceptual decision-making processes will be similar when processing a more complex and ecologically meaningful stimulus, such as natural actions, is an open question. This study aimed to investigate this gap in the literature by examining the neural basis of perceptual decisions related to the perception of natural actions. To this end, in this study, which included three separate EEG sessions, twenty participants performed a discrimination task between action exemplars belonging to a different action class in each session (either one of the locomotion, skin-displacement actions, or manipulation actions). To control for task difficulty, the same coherence manipulation was applied to each action exemplar to create four levels of coherence, and behavioral data analysis showed that mean response times and miss rates decreased as the level of coherence increased. Furthermore, the results of event-related potential analysis showed that the CPP signal followed the level of coherence of the stimulus within all action classes, which is in line with the literature, and also that the manipulation of the action class had a significant effect on the CPP signal. In contrast to CPP, the LRP signal showed an independent build-up from both the strength of the sensory evidence and the identity of the action class. Taken together, the findings of the current study support the generalizability of perceptual decision-making stages and their neural basis, which are defined by utilizing much simpler stimuli, to decisions related to the perception of natural actions while revealing that the identity of natural movements significantly influences the decision process.