Modeling and monitoring of engagement and affect in human-computer interaction systems
Is this your thesis?
This record came from a bulk archive import. If it’s yours, link it to your profile.
Abstract (EN)
Social robots are becoming widely used in human-computer interaction (HCI) systems with their artificial intelligence powered skills. Modeling and monitoring of engagement and affective state in the course of social HCI set important problems. In this thesis, we investigate three aspects of these important problems that are monitoring engagement and affective state in HCI settings, and affective talking head generation for more natural HCI applications. Monitoring of engagement has been studied as a part of the joint work on constructing multimodal database of engagement in human-robot interactions (eHRI Database). The second aspect is affective state monitoring in real-time HCI settings. We construct a CNN-GRU architecture to estimate affective state attributes from the speech signal. The proposed affective state estimation model has been trained with RECOLA and CreativeIT databases and tested on the IEMOCAP database. Lastly, we investigate contributions of speech and facial landmarks in talking head generation. We find that landmark based models have better performance compared to speech based models. However, speech can be used as an additional input along with landmarks to improve the performance further.
Author
Ege Kesim
How to Cite
Ege Kesim (Master Thesis). Modeling and monitoring of engagement and affect in human-computer interaction systems, 2021, Koç University.
Keywords
License
Tüm Hakları Saklıdır
This work is shared under the specified license terms.
More theses from Koç University
- Thinking of the ECtHR case law on same-sex marriage from the perspective of modern natural law: An evaluation based on the right to marry under the ECHR(2023)
- Obje tabanlı akıl danışma-tavsiye iletişimi tasarımına ilham kaynağı olarak Türk kahve falı(2017)
- The Making of a medical museum: Recontextualizing the Gülhane medical history collection in Ankara, Türkiye(2024)
- State-building in multi-ethnic borderlands: Nationalizing Eastern Anatolia and Transylvania in interwar Turkey and Romania(2021)
- Unwritten histories of photography: Elisa Zonaro, an Italian photographer in Ottoman Istanbul(2023)
- CNCmachine tool digital twin applications with advancements of edge computing(2024)
