Vision based positioning ABB IRB 140 robot for gas leakage test automation

Technological progress has been recorded in factories over the past decade. Dangerous, repetitive and difficult tasks on production lines are carried out by automatic machines instead of people. The industrial robots provide significant flexibility and adaptability in production lines. With robotic working conditions, increased productivity and quality, independent production from the human labor force will take place. There will be integrated factories that are compatible with the Factory of Futures. The factories will use robots in a variety of tasks, and many research and development, performance improvement and complex control algorithms for robots are still being developed. It begins with the use of industrial robots in General Motors in 1961 for use in production lines. The first patent of industrial robots was taken by George Devol in 1965. Today, the number of industrial robots is increasing, especially in the automotive sector. By the end of 2017, a total of nearly 2 million industrial robot are being used. One of the benefits provided by the using robots is that movements are reproducible, accurate and reliable. However, it has been seen that robots do not work successfully in situations that change if they work on their own. According to the changing conditions, manual teaching will not be a solution because it takes a long-time procedure and processes that affect it completely. For this reason, it is one of the most important issues to be notified about the environment of robots in the automation processes, examples of which are vision based positioning robots. Computer vision applications contains methods for processing, analyzing and understanding images. High-dimensional data from the real world is often used to generate numerical and symbolic information with decision forms. The increasing use of these applications is a perception of the underlying vision of a computer vision. Computer vision applications are used in industry, medicine and robot technology. Vision based robot positioning started to be used in the early 1970s. Many working systems and approaches have been proposed theoretically and technically in image processing such as positioning and depth adjustment, feature selection and processing, camera models and calibration. In this thesis, firstly, a 6-axis ABB IRB 140 robot which are 3 are translational and 3 are rotational for robotic automatic test system is presented. ABB IRB 140 robot consists of 5 main parts. The first one is the controller of the robot where the robot is programmed and the drivers are found. The second one is robot arms where robot movements are made. The robot determines the mobility, its accessibility, and the maximum capacity according to the size and shape of the robot arms. The third is the drivers that determine the speed and acceleration of robot movements. The fourth is the gripper which is added to the tip of the robot and changes its relative position. A gas leak detector was used in this study. Finally, the sensors measure the speed and acceleration of the robot. Robot movements and target positions are defined in coordinate systems. Robots allow to work in many coordinates. These are base, world, work object and tool coordinate systems. The target position of the robots is determined according to the final position of the vehicle attached to the robot end effector, and this point is called the tool center point. The world coordinate system is used as the coordinate system in ABB IRB 140 robot. The kinematic and dynamic models of ABB IRB 140 are given. In the kinematic model of the robot there is a simplification of the mathematical Denavit Hartenberg algorithm. In this model, the robot calculates the position of the robot center point in the coordinate system depending on the joints. In the dynamic model, the interaction of the joints expressing the interaction of the joints is given. The ABB IRB 140 robot is programmed using the RAPID programming language. Robot relates to TCP-IP connection protocol and programmed in RoboStudio program on robot computer and the program of robot is installed. The ABB IRB 140 robot works in this project with an automation system in which the PLC and camera are controlled PC module. For this reason, the robot has been implemented with PLC. Robot receives all signals and information from PLC via Profinet communication protocol. In the second stage of the thesis study, image processing algorithms are explained and coordinate transformation and camera calibration are done. Digital image processing algorithms have been used in this study. These methods are threshold, morphological operators and color plane extraction methods. In the threshold method, segmentation is performed by assigning two or more classes of each pixel in the source image to the color images from the grayscale image. This method divides the image intended for processing by using at least one color or gray scale value. Morphological operators are an effective image processing method in nonlinear neighborhood processing. In morphological operators which are dilation, erosion, opening and closing operators are used. The size of the objects is enlarged in the dilation operator, and the size of the objects is reduced in the erosion operator. In the opening operator, objects close to each other are removed without any change, while objects in close operators are connected to each other. With the color plane extraction method, the color information is obtained. The positions obtained in the image processing method are included in the image coordinate system. However, the positions in the ABB IRB 140 program are defined in the world coordinate system. For this reason, coordinate transformation is performed with linear equations. Subsequently, the camera calibration for the coordinate transformation was done on a grid plane and the parameters were found. These parameters have been updated according to the depth change in the algorithm. At the next stage of the thesis work, a gas leakage test is carried out using robotic automation. In the heat pump dryer, the air in the drum has air flow to raise the air temperature and to extract the air from the ambient air, with the heat pump compressor. The heat pump dryer consists of compressor, condenser, evaporator and copper pipes connecting them. There are welding points on the copper pipe where the components are connected to each other. In this study, these welding points will be leak tested with ABB IRB 140 robot. Two types of tests are presented, which are the high pressure test and the equilibrium test. Firstly, a high pressure test will be performed. In this test, four welding points on the copper pipe leading to the condenser from the compressor are tested. The test is done when the gas in the pipe is between 15-20 bars behind the compressor stop. Secondly, the equilibrium test is performed. This test is performed by testing the 3 welding points of the pipe coming out from the compressor to evaporator. The test is made 10 seconds after the compressor stops and the high pressure test. The gas pressure in the pipe goes up to 9 bar. The positions of the welding points are varied due to various reasons. In this thesis, ABB IRB 140 robot system with vision based positioning is presented to provide the flexibility of correcting the gas leakage tests with robotic automation. The vision based positioning is to obtain position information from the image processing algorithm of the target robot. In this study, a single camera was used and the camera is mounted in a fixed position. In addition, in this thesis, the area of the welding points is divided into 3 according to the depths of the points and image processing algorithms are formed for each area. Algorithms re done with the NI Vision Assistant application and run in sequence in the Labview program. In the three image processing algorithms, mask operations are used in the original image to initially constrain the restrictive rectangle. The threshold operator is used to make the copper pipe visible. The threshold operator parameters are 72, 82 and 93, respectively. After the threshold method, the morphological operators are applied more than once in the same algorithm to protect the welded pipes and to sift the images with threshold values close to the copper pipe. After the copper pipe was obtained, the Lookup table was applied to the image obtained to change the dynamic intensity of the field in the image. After applying the lookup table, the original image is overlaid with the image obtained in the algorithm after the color plane of the image is extracted. Finally, it has been applied to three image processing algorithms to determine the welding points in the field with the maximum clamping operator. Finally, for demonstrating the performance of the gas leakage test with ABB IRB 140 robot with the vision based positioning, real-time experiment work has been carried out. The real-time experiments were performed by collecting the position data of 500 welding points and graphically and statistically examining the position data. The experimental results showed that the ABB IRB 140 robot application with image-based positioning was successful in capturing position deviations and performing the test. In the experiments, parameter update was done firstly according to the depth value in image processing areas. Secondly, the deviations of the welding points in the high pressure test have been investigated. In this test, it is seen that the deflection amount at the 6. and 7. welding points is more in the X axis. Then, the amount of deviations of the 1. and 2. welding points is less in the X axis. In the high pressure test, there is a minimum deviation of -187.8 mm and a maximum of 165.5 mm on the X axis. For this reason, it was observed that the ABB IRB 140 robot with vision based positioning on the X axis is successful. In the high-pressure test, the deviations of the 4 welding points are observed on the Y-axis but less than the X axis. At the welding points, a minimum deviation of -76.49 mm and a maximum of 92.43 mm was observed on the Y-axis. Third, the position deviation values of the welding points in the equilibrium test are examined. In this test, the amount of deviations of the 5. welding point is found to be larger that 3. and 4. welding points on the X axis. the amount of deviations on the X axis was observed to be less for at the 3. and 4. welding points. There is a minimum deviation of -122.8 mm and a maximum of 93.26 mm on the X axis. For this reason, it was observed that the ABB IRB 140 robot with image-based positioning on the X-axis was successful. At these welding points, the minimum deviation is -76.76 mm and the maximum deviations 77.03 mm was observed on the Y-axis. In summary, ABB IRB 140 robot application with vision based positioning for gas leak test has been realized in this Master thesis. In this study, a 6-axis ABB IRB 140 robot is used, the image processing algorithms are created to find the welding points on the copper pipe, the coordinate transformation is made between the camera and the robot, and position deviations are examined in real time applications to demonstrate the success of the application.