Determining the position of olive fruit using image processing methods for robotic systems

With the advancement of technology, the increase in image processing techniques and the use of artificial intelligence has made it possible for processes to be automatically carried out in many fields. For example, in the past few years, in addition to agricultural pest control and irrigation processes, challenging processes such as fruit harvesting have also been automated. In automatic harvesting systems, it is necessary for the robot responsible for harvesting to be able to identify and approach the fruit by determining its coordinates. A data set that will be used for modeling the system is a very important requirement for accurately identifying the fruit. There are many studies in the literature on automatic fruit harvesting and the automatic detection of fruit. However, there is no study on the detection of olive fruit, which is densely found on tree branches and almost the same color and size as the leaves, in real environments. In this study, a dataset was created from images of the fruit of an olive tree obtained using 12 MP Optical image stabilizer (OIS) camera and an IP camera with a 1.3 MP CMOS photo sensor. Then various image processing techniques and fruit labeling processes were applied to the images in the dataset. Finally, the process of detecting olive fruit in real environments was carried out using CNN, YOLOv3-tiny, and YOLOv4-tiny-custom deep learning models. It is also necessary to accurately measure the distance between the robot arm and the fruit in order for the robot arm to be able to access the olive fruit. For this reason, in the study, Microsoft Kinect v1 and stereo camera configurations were also used to detect the depth of the fruit. According to the results of the models used to detect olive fruit, the YOLOv4-tiny-custom model achieved the highest success rate with a 0.88 F1 score.