Data mining based novel approaches to multiagent reinforcement learning

ABSTRACT PhD Thesis DATA MINING BASED NOVEL APPROACHES TO MULTIAGENT REINFORCEMENT LEARNING Mehmet KAYA Fırat University Graduate School of Natural and Applied Sciences Department of Electrical and Electronics Engineering 2003, Page: 91 Reinforcement learning is considered as a strong method for learning in multiagent systems environments. However, it still has some drawbacks, including modeling other learning agents present in the domain as part of the state of the environment, and experiencing some states more than needed during the learning phase. On one hand, although some states are not experienced sufficiently, it is expected that an agent has to select an appropriate action in each state. On the other hand, before the learning process is completed, it is not given permission for an agent to exhibit a certain behavior in some states that may be experienced sufficiently. This causes the increment of the learning time. This case shows that learning in a partially observable and dynamic multiagent systems environment still constitutes a difficult and major research problem that worth further investigation. In order to handle the problems mentioned above, in this thesis, we propose novel multiagent learning approaches for a cooperative learning system. Our approaches incorporate fuzziness and online analytical processing (OLAP) based data mining to effectively process the information reported by the agents. First, we describe a fuzzy data cube OLAP architecture which facilitates effective storage and processing of the state information reported by agents. This way, the action of the other agent, even not in the visual environment of the agent under consideration, can simply be estimated by extracting online association rules, a well-known data mining technique, from the constructed data cube. Second, we present a new action selection model which is also based on association rules mining. Then, we generalize states which are not experienced sufficiently by mining multiple-levels association rules from the proposed data cubes. Finally, we present a new and robust modular architecture and a corresponding learning approach to overcome the various problems encountered by the agents in multiagent environment, such as slow convergence speed and the number of steps to convergence. Then, we successfully combine advantages of the modular approach with all of the features described above. Results obtained for a well-known pursuit domain show the robustness and effectiveness of the proposed mining based learning approaches. Keywords: Multiagent reinforcement learning, data mining, association rules, online analytical processing, fuzzy sets. VII