Theses supervised by Dr. Öğr. Üyesi Reyhan Aydoğan
10 theses · Özyeğin University
Familiarity effect on human-agent negotiations
Artificial Intelligence has changed our world in various ways. People have started to interact with a variety of intelligent systems on a daily basis. As the interaction between human and AI systems increases day by day, the factors influencing their communication have become more and more important especially in the field of human-agent negotiation. Therefore, it is necessary to study the factors affecting human-human negotiation while designing agents negotiating with their human counterparts. As familiarity is one of these factors, this work aims to investigate the effect of familiarity on human-agent negotiation so that we can design more effective negotiation systems. Being familiar to other party may influence how we interact and hence the process and outcome of the negotiation. Our hypothesis is that negotiating with a familiar opponent would create a difference in terms of negotiation process and outcome. In order to study this effect in human-agent negotiations, we developed negotiation framework in which human participants negotiate with an avatar in a bilateral fashion. To measure the effect of the appearance familiarity in negotiation, two control variables are defined: negotiating with a celebrity avatar and negotiating with a non-celebrity avatar. In order to avoid the learning effect, we adopt a between-subject experiment design. We recruited 67 participants and analyzed their negotiation data elaborately as well as their subjective opinions specified in the questionnaires. Our experimental results showed that being familiar with the opponent affected both negotiation process and outcome. Particularly, human participants had a tendency to be more collaborative when their opponent is a celebrity avatar versus a non-celebrity avatar.
Tedarik zincirleri için etmen bazlı bir pazarlık yöntemi
A supply chain is a network enabling the flow of goods from suppliers to end customers. In a supply chain, suppliers, manufacturers, distributors and retailers interact with each other to buy, process, store and sell products. In an efficient supply chain the main aim is to minimize the costs while maximizing the profits. To achieve this, the supply chain entities interact with each other wisely. For example, they need to agree on contracts maximizing their profits but not causing infeasibilities in the delivery or payment. In this thesis we worked on a supply chain simulation, namely the supply chain world, which has been used in Supply Chain Management League (SCML) of Automated Negotiating Agents Competition (ANAC), an international supply chain competition. In the supply chain world simulation environment, we proposed a factory manager agent which can procure products from other agents, processing some of them, and sells it with profit to other agents. The interactions with other agents are done in closed bilateral negotiations protocol. The motivation of the proposed agent is determining a good level of altruism and greediness in negotiations depending on the supply and demands of products. To evaluate the performance of the proposed agent strategy, we run a number of simulations and compare the performance of our agent with the performance of the top performing agents in the SCML 2019. The experiment results show that our agent, MyGreedyAgent, managed to adjust its greediness and altruism effectively based on the supply demand ratio and performed as well as ANAC 2019 top agents.
Otomatikleştirilmiş ikili pazarlıklarda ilişkisel ve frekansçı rakip modelleme yaklaşımları
This thesis mainly focuses on the problem of learning opponent's preferences during the negotiation in bilateral automated negotiation in which agents negotiate with each other to reach an agreement. Accordingly, it addresses the problems with the classical frequentist approach and advances the state-of-the-art in opponent modeling in automated negotiation by introducing a novel frequency opponent modeling mechanism, which updates some of the assumptions introduced by classical frequency approaches. Moreover, this thesis also proposes adopting association rule mining techniques to learn the opponent's preferences in bilateral negotiation. An extensive evaluation of those proposed approaches shows that the proposed approaches outperform the classical frequency model. In addition, this thesis argues that while optimizing one's utility function is essential, agents in a society should not ignore the opponent's utility in the final agreement to improve the agent's long-term interests in the system. It aims to show whether or not it is possible to design a social agent (i.e., one that aims to optimize both sides' utility functions) while performing efficiently in an agent society. Accordingly, we propose a social agent supported by a portfolio of strategies, a novel tit-for-tat concession mechanism, and a frequency-based opponent modeling mechanism capable of adapting its behavior according to the opponent's behavior and the state of the negotiation. The results show that the proposed social agent does not only maximize social metrics such as the distance to the Nash bargaining point or the Kalai point but also is shown to be a pure and mixed equilibrium strategy in some realistic agent societies.
İki taraflı müzakere için aktör-kritik pekiştirmeli öğrenme yaklaşımı
Designing an effective and intelligent bidding strategy is one of the most compelling research challenges in automated negotiation, where software agents negotiate with each other to find a mutual agreement when there is a conflict of interests. Instead of designing a hand-crafted decision-making module, this thesis proposes a novel bidding strategy adopting an actor-critic reinforcement learning approach, which learns what to offer in a bilateral negotiation. An entropy reinforcement learning framework called \acrfull{sac} is applied to the bidding problem, and a self-play approach is employed to train the model determining the target utility of the coming offer based on previous offer exchanges and remaining time. Furthermore, an imitation learning approach called behavior cloning is adopted to speed up the learning process. Also, a novel reward function is introduced that does not only take the agent's own utility, but also the opponent's utility at the end of the negotiation. The developed agent is empirically evaluated. Thus, a large number of negotiation sessions are run against a variety of opponents selected in different domains varying in size and opposition. The agent's performance is compared with its opponents and the performance of the baseline agents negotiating with the same opponents. The empirical results show that our agent successfully negotiates against challenging opponents in different negotiation scenarios without requiring any former information about the opponent or domain in advance. Furthermore, it achieves better results than the baseline agents regarding the received utility at the end of the successful negotiations.
Effect of embodiment in human-agent negotiations
With the current advancement in artificial intelligence, intelligent systems interacting with humans are becoming more prevalent in our lives. One of the challenges is building socially intelligent agents who can effectively negotiate with their human counterparts. While designing such agents, it is essential to consider humans' bounded-rationality, emotions, facial expressions, arguments, and gestures. Apart from this, the perception of a physically embodied agent might be different from the virtually embodied one. It might influence human negotiators' decisions and responses. Accordingly, this thesis focuses on the effect of bidding strategy, gestures, and embodiment in human-agent negotiations where the human participants negotiate with a humanoid robot. Our experimental results show that taking the human opponent's bidding behavior into account while generating an offer improves the robot's negotiation outcome. Another important observation is that an agent designer should be careful while designing gestures compatible with the agent's behavior since the gestures may also negatively affect the negotiation result. Furthermore, the experimental results showed that the social welfare was higher when the negotiation was held with a virtual robot rather than a physical robot. We observed that human participants made more collaborative moves during the negotiation in the virtual setting compared to the physical environment.
Negotiation-based decentralized conflict resolution in multi-agent path finding
This thesis addresses the problem of Multi-Agent Path Finding problem where multiple agents aim to reach their destination in a grid world without any colli sion. It aims to provide a solution achieving good trade-off between the privacy of the agents and the effectiveness of solutions. Accordingly, a token-based bilateral negotiation approach is presented to solve this problem in a distributed way. The proposed approach is evaluated empirically in various scenarios by comparing it with state-of-the-art centralized approaches such as Conflict Based Search and its variants. The experimental results showed that the proposed approach can find conflict-free path solutions albeit suboptimally, especially when the search space is large and high-density, whereas centralized approaches struggle to find optimal solutions. Despite being outperformed by suboptimal centralized solvers, the pro posed decentralized approach can achieve considerable results with naive agents by sharing minimal information about themselves. The proposed approach also enables agents to have their autonomy; thus, the proposed approach is convenient for MAPF problems involving self-interested agents.
Considering arguments in human-agent negotiations
Autonomous negotiating agents, which can interact with other agents, aim to solve decision-making problems involving participants with conflicting interests. Designing agents capable of negotiating with human partners requires considering some human factors, such as emotional states and arguments. For this purpose, we introduce an extended taxonomy of argument types capturing human speech acts during the negotiation and propose an argument-based automated negotiating agent that can extract human arguments from a chat-based environment using a hierarchical classifier. Consequently, the proposed agent can understand the received arguments and adapt its strategy accordingly while negotiating with its human counterparts. We initially conducted human-agent negotiation experiments to construct a negotiation corpus to train our classifier. According to the experimental results, it is seen that the proposed hierarchical classifier successfully extracted the arguments from the given text. Moreover, we conducted a second experiment where we tested the performance of the designed negotiation strategy considering the human opponent's arguments and emotions. Our results showed that the proposed agent beats the human negotiator and gains higher utility than the baseline agent.
A comprehensive human-agent negotiation framework: Preferences, emotion & interaction
In today's increasingly interconnected world, human-agent negotiation plays a pivotal role in reaching socially beneficial agreements when stakeholders need to make joint decisions. Developing intelligent agents capable of understanding not only human negotiators' preferences but also attitudes is a significant prerequisite for effective human-agent interactions. Awareness of a human's emotional state and ability to express an agent's mood to influence the human negotiator might significantly affect the negotiation outcome. This thesis presents a comprehensive framework that revolutionizes the field of human-agent negotiation, integrating two critical elements: Emotionally aware negotiation strategy and Conflict-Based Opponent Modeling (CBOM). By combining these novel approaches, the framework enhances negotiation outcomes and fosters cooperation between agents and human negotiators, ultimately leading to mutually advantageous agreements. The thesis establishes the research context and motivation, underscoring the escalating importance of human-agent negotiation in a world where collaborative decision-making is essential for addressing complex challenges. It highlights the need for advanced agents to accurately interpret human preferences and behaviors, enabling admissible settlements that serve joint interests. Shedding light on the limitations of conventional approaches that heavily rely on opponent offers and remaining time. Additionally, it explores the critical role of emotional awareness and opponent modeling strategies in human-agent negotiation. The synthesis of existing research lays the groundwork for developing the proposed comprehensive framework. Emotional awareness takes center stage in the proposed negotiation strategy. Solver Agent: Emotional Extension of the Hybrid Agent bidding strategy is introduced. The Solver Agent considers the opponent's emotional state during negotiation, leading to higher social welfare scores and faster agreement times. The experimental study emphasized the profound impact of emotional awareness on negotiation outcomes, particularly in human-agent settings. CBOM efficiently extracts maximum information from limited interaction rounds in human-agent negotiation settings, surpassing traditional approaches in prediction performance. Experimental analyses confirmed the superiority of CBOM in human-agent and automated negotiation scenarios, even when the exploration of the outcome space is limited. The experimental findings establish CBOM as a powerful tool for modeling human behavior and preferences in negotiation. In conclusion, the comprehensive human-agent negotiation framework presented in this thesis represents a significant advancement in the field. By seamlessly combining Conflict-Based Opponent Modeling and Emotional Awareness, the framework empowers intelligent agents to discern human preferences and behaviors more accurately, facilitating cooperative interactions and achieving mutually beneficial agreements. The framework's effectiveness in human-agent and automated negotiation settings highlights its potential for designing negotiation agents that interact adeptly with human negotiators, fostering understanding and optimizing negotiation outcomes. The future of human-agent negotiation lies in forging a new era of cooperation, where intelligent agents serve as capable partners, promoting social welfare and driving positive change through admissible settlements that incorporate joint interests. This thesis contributes valuable insights towards realizing this vision, marking a significant step forward in the field of human-agent interaction.
Towards transparent recommenders: An explanation-based negotiation approach
As more and more recommendation systems are used in different areas and they are exposed to more ethical concerns, there is a growing demand for transparent and persuasive interactions with these systems. Toward this end, incorporating explainability in recommendation systems has emerged as a promising approach to enhance sociability and user trust. This thesis focuses on recommendation systems that utilize explainability techniques to foster sociability by providing precise and understandable explanations for their recommendations. The proposed recommendation system utilizes a combination of data-driven transparent mechanisms and human-agent negotiation approaches. The system generates personalized recommendations based on individual preferences and other similar user-tailored factors and engages in a negotiation with the users via discussions through explanations and real-time feedback mechanisms. The system reacts to user responses online, tailoring subsequent recommendations and explanations to convince the user. This thesis encompasses Nutrition Virtual Coach (NVC) agents that generate personalized food recommendations based on individual factors like allergies, eating habits, lifestyles, and ingredient preferences. It mainly focuses on explanation generation techniques to enhance the transparency and trustworthiness of the system by improving the NVC agent's sociability in multiple steps. Ultimately, we incrementally conducted multiple experiments with participants from various backgrounds to evaluate the acceptability and effectiveness of the system. The findings from the experiments generally indicate that most participants appreciate the opportunity to provide feedback and receive explanations for the given recommendations. The participants prefer receiving information tailored to their specific needs and expectations. Additionally, the participants expressed their thoughts on various forms of explanations. The findings indicate that comparative explanations are not appreciated as much as informative explanations. The users seem to prefer direct and simple explanations that explain items respectively.
İkili müzakerelerde zaman serisi tahmin modelleri
This thesis explores the dynamics of agent-based negotiations, with a focus on understanding opponent's behavior and predicting their offering patterns to make strategic decisions. Guessing the utility of the opponent's upcoming offers valuable insights for the agent's subsequent moves. The research aims to predict the opponent's future offers by employing diverse learning algorithms in various situations to measure their effectiveness in comprehending negotiation behavior. The prediction study comprises two parts; one investigating the impact of these models in one-to-one negotiations, specifically tailored for the agent's own experiences, while the other examines the performance of predictive models in a tournament setting for all agents. A learning process with three distinct targets have been established to assess the prediction models: (i) estimating the agent's utility of the opponent's next offer by considering only its offer history, (ii) estimating the agent's utility considering opponent-related variables, and (iii) estimating the opponent's utility using opponent-related variables. According to the experimented results, the best learning approach is incorporated into an agent design to observe the impacts of having predictions of future utility values on the agent's negotiation success. The thesis evaluates these models in diverse negotiation scenarios and highlights promising outcomes for the proposed methods. It also introduces a novel negotiation strategy called `Negoformer', which incorporates predictions into the offering strategy and investigates their impact on the outcome of negotiations. The experiments showcased the success of Negoformer compared to other agents in various negotiation success metrics, such as individual utility value and social welfare score.