Manipulation and placement planning for loading a dishwasher by a robot

Placing the dishes into a dishwasher can take a considerable amount of time, which is a process a person would have to repeat every few days. Therefore, we decided to build a robotic system which would place the mugs on the counter-top to the dishwasher tray. Three types of problems are solved in order to realize this task: object recognition for detecting the mugs; 2D packing problem for placement planning of mugs on the dishwasher tray; placing the mugs into the dishwasher through manipulation. These tasks are solved within a simulation environment and in the physical world by a 5 degree-of-freedom robot arm and a depth camera. Our contribution with this project is placement planning, in which we use different heuristics and optimization methods with different cost methods and functions. Top-Left-Fill (TLF) and Left-Top-Fill (LTF) heuristics are used for packing and Simulated Annealing (SA), Genetic Algorithm (GA), and two different Particle Swarm Optimization (PSO) methods are used for finding the near-optimal solution for the placement of the mugs. The initial configurations of our methods are made with closest-item-first, sorted-size heuristics and random order. In our optimization algorithms, we use weighted-sum (WS) and ranking fitness (RF) methods with bin-packing cost (BPC), user preference-based cost (UPC) and engineering cost (EC) functions. In order to find the parameters and their weights for the cost functions, we conducted preliminary and user studies. Furthermore, we made an observational study to examine the placement methods of the participants. An annotation system was made to rate the sortedness of these placements with a Likert scale. The sortedness of the placements were compared with the Obsessive-Compulsive Scale test results from the observational study to observe the relation. The results of our simulation experiments show that, in general, the GA with the LTF heuristic produces the lowest costs with the WS method, whereas for the RF method, the best optimization method is the SA.