Vision-based robotic plant growth monitoring and learning in polyculture farming

This thesis is focused on vision-based robotic plant growth monitoring and learning in polyculture farming. This type of agriculture dictates special soil preparation and farming a variety of companion plants together and hence differs from monoculture agriculture. The three-axis cartesian robot TarRob endowed with vision is used. First, its workspace is set up as an indoor polyculture garden through preparing the soil and planting a variety of companion plants. The behavioral capabilities of TarRob are extended to collect RGB-D images of these plants regularly. A polyculture plant dataset is initiated by adapting a four times daily schedule six hours apart over three months. Next, learning-based models for polyculture plant detection, plant segmentation, and leaf detection have developed. These models are primarily YOLO-based models that are subjected to further training. For this, the dataset is expanded as to include ground truth segment and leaf data. Following, the computation of the structural phenotypes is considered. A three-dimensional model of each plant is constructed via combining RGB-based detection and segmentation results with the depth data and is then used to compute the major structural phenotypes including height, width, length, area, and leaf count. Finally, the temporal nature of these phenotypes is studied from the resulting time series data. In particular, various statistical methods and learning models can be used to learn how each plant type grows. This thesis presents a comprehensive, fast, and modular approach to the automated monitoring and analysis of plant growth in polyculture farming by integrating robotics with machine vision.