Real-time intelligent strawberry harvesting and quality determination system using computer vision and deep learning

Strawberries have a comparatively extended harvesting period, which poses the need for intelligent harvesting systems that identify various stages of strawberry ripeness to alleviate fatigue and reduce the costs associated with this task. These systems offer potential solutions to enhance productivity while minimizing labor requirements. Research in real-time strawberry detection still has several gaps to address. These gaps include managing imbalance label distribution, exploring data augmentation techniques, optimizing preprocessing and training parameters, and investigating advanced topics such as fine-tuning. This research focuses on developing an accurate and efficient real-time strawberry detection model. The main objective is to locate and identify strawberries in agricultural orchards and assess their quality by detecting overripe and decaying fruits. For this purpose, a diverse High-Quality Annotated Strawberry Dataset(HQASD) was collected with 5000 RGB images belonging to ten strawberry maturity levels. Different types of data augmentation techniques were applied, and the most advanced and novel approach used in this context was the Cycle-Consistent Generative Adversarial Network (CycleGAN) for generating images of an underrepresented class from an overrepresented one. The latest real-time object detection model, You Look Only Once (YOLOv7), is trained on real and synthetically generated data. Deep experiments and various scenarios were conducted to investigate all the aspects that impact the model's overall performance. Hyper-tuning is performed on the model's parameters to optimize its performance. The best-obtained models for identifying ten classes achieved Mean Average Precision (mAP) 98.7% at Intersection over Union (IoU) threshold 0.5, and the best model for identifying three classes scored 96.51% mAP for all types at the same threshold, where the model trained on synthetic images performed 98.4% mAP for all classes which shade the effectiveness of employing CycleGAN along with fine-tuning techniques. In addition, the research succeeded in creating HQASD, which provides a comprehensive representation of the strawberry maturity spectrum, which makes it well-suited for computer vision tasks.