Reinforcement learning based distributed fault diagnosis system for autonomous robots

Autonomous mobile robots (AMRs) are one of the most essential cyber-physical systems for industry 4.0. Faults of these robots can cause accidents, delays, and interruptions in production. Data-driven artificial intelligence methods are successful in detecting faults. However, training successful artificial intelligence models requires extensive data representing different systems. Sending raw sensor data to the central system poses speed, security, and bandwidth issues. On the other hand, models trained only on edge systems do not represent all agents and state space. Centralized and synchronized training for multiple mobile robot agents is difficult in real and dynamic environments such as intelligent factories. Every agent faces unknown difficult situations, and faults are rare, which brings the problem of training the model under unbalanced conditions. This thesis proposes a fault detection method based on multi-agent reinforcement learning for AMRs operating in fleet order on intelligent factory floors for prognostics and health management. The method is based on asynchronously sharing model parameters between systems instead of transferring raw data from edge systems to the central system. In model training, imitation learning was used, which allows learning about human experience. The study proposes a parameterized proportional reward mechanism for unbalanced data in training multiple agents in unknown environments. The models perform sensor-based anomaly detection at the edge. During the fault detection phase, decision-level fusion is applied against noise that can be seen in a single sensor data set and causes false alarms. The study was conducted at Eskişehir Osmangazi University Intelligent Factory and Robotics Laboratory. The results show that the proposed method eliminates all false alarms for normal runs. In addition, faulty runs were detected at a rate of 100%, and faulty windows were detected at approximately 78%. The proposed method can be used in accurate maintenance planning by detecting the moment of fault with high accuracy.