Plant and phenology recognition from field images using texture and color features

In today's world, factors such as global warming, population growth, drought, erosion and the use of agricultural fields for different purposes have made the issue of agricultural efficiency important. Today's information technology is needed to advance agricultural efficiency. The data collected as a result of the monitoring of agricultural areas through sensors and cameras leads to productivity-enhancing applications on agricultural terrain. There are national agricultural monitoring networks in many countries around the world. TARBIL agricultural monitoring and information system was established and supported by the Ministry of Agriculture, Food and livestock in Turkey. In many parts of Turkey, agricultural areas have been furnished with ground stations that accommodate sensors and cameras. Meteorological and image data collected from these stations are transmitted instantly to a server. In the current system, manual expert analysis is needed to determine the plant type and phenological stage information from the collected images in order to label them for archival use as well as for research related to the development of irrigation, fertilization and disinfection regimes and prediction of yield. In this work, we consider the utility of automatic image classification since manual classification is time consuming and hinders agricultural efficiency. In this thesis study, a system was developed to automatically detect the plant type and phenological stage by using 13323 close-up field images in 1x and 10x zoom type levels of barley, sunflower, wheat, corn, chickpeas and cotton plants taken in 66 different observation periods at 58 different TARBIL stations. The texture and color features of the images were used to solve this problem. Methods of extracting texture features are Histogram of Oriented Gradients (HOG), Gray Level Co-occurence Matrices (GLCM) and Law's Texture Energy Measures (LTE)[6-9,12]. The color feature extraction method used is Color Moments [11]. Histogram of Oriented Gradients (HOG) is a method used to extract object-related features by looking at edge information and gradient values of objects in the images. In this method, the image is divided into small local regions called "cells" and these regions are defined by local histograms as identifiers. The local histogram refers to the distribution of the number of occurrences of the gradient vectors calculated for the cells in the specified direction range. HOG descriptor operates on localized cells, so the method is invariant to geometric and photometric transformations and is not affected by varying lighting conditions. In this study, each image data is divided into 100 (10x10) cells and a normalized local histogram is created for each cell. The histograms in the 100 cells are combined to yield a (10x10x9) = 900 dimensional HOG features. The principal component analysis (PCA) method is used to reduce the number of operations to be performed because the features obtained by the HOG method are having high dimensions. Gray Level Co-occurrence Matrices (GLCM) is a statistical method for extracting the texture properties of images. GLCM expresses the frequency of occurrences of a certain pair of pixel pairs with i and j gray level values with a certain geometric distance between them and is a square matrix with each dimension equal to the number of different gray level values (N) in the image. When gray level co-occurrence matrices are calculated, the relation between two pixels is determined according to distance and direction information. The obtained co-occurrence matrices are used to obtain statistical information to define the texture. In this study, (4x5) = 20 features were extracted from these statistical data in terms of contrast, homogeneity, entropy, energy and dissimilarity for the 0◦, 45◦, 90◦, 135◦ directions. Law's Texture Energy Measures (LTE) is based on the use of four predetermined one-dimensional vectors to create two-dimensional convolutional masks that can be used on the image to compute the image's texture energy. The obtained two-dimensional convolution masks are convolved with the images to obtain texture energy maps. In this study, 9 texture energy maps are used. By averaging values of these maps 9 features are extracted. Color moments, a method for extracting color features and calculating the similarity of color distributions of images, take into account the dominant features of colors, rather than considering all color information in the images. These dominant features are the mean, standard deviation and skewness moments, which are the first three moments of probability distributions of colors. In this work, color moments are extracted after converting images from RGB color format to HSV color format so that lighting conditions such as light, shadow, background do not affect the results. In each image, 9 color attributes are extracted by calculating 3 different moments for each of 3 channels in the HSV color format. The color features obtained in this study were classified by Naive Bayesian, k-nearest neighbor, support vector machines and random forest classifiers after being combined with each texture feature. Parameter optimization for support vector machines and k-nearest neighbor classifiers improves the success of the classification results. In order to train the classification before the classification procedures, it is necessary to create a training set and a test set to test the model created after the training. In this study, 5-fold cross validation and leave-one-out cross validation method was used to construct training and test sets. Accuracy success rates of texture analysis methods used with color analysis methods were measured with different classifiers. From the results, it was seen that the effect of the preparation method of the training and test data set on the classification performance was great. Classification performance by 5-fold cross validation method is considerably higher than the classification performance by leave-one-out cross validation method. The results show that the HOG method gives the best performance with the support vector machine classifier. In general, the lowest performance in classifiers is achieved with the Naive Bayes classifier. The system proposed in this study can be used for the automated plant and its phenological stage recognition process on the field with Tablet PCs to be given to farmers. Farmers can then make inferences about irrigation, fertilization regime determination and yield prediction. In addition, automatic labelling of images captured by cameras in many stations before archiving facilitates research on the prediction of irrigation, fertilization regime, spraying and yield. The results of this study will also be used as a benchmark to assess the performance of the methods to be developed with deep learning.