Computer visión and artificial intelligence for yield components' assessment in digital viticulture stars

Abstract

Grapevine yield components’ estimation is a highly relevant task for the wine industry and grape growers. Traditional methods to obtain these components are usually tedious, time-demanding and limited to a reduced acquisition of data. New technologies have arisen to offer an alternative to these methods. Among them, computer vision and artificial intelligence have been widely used in different fields to transform the information contained in images and other data into knowledge that supports decision-making. Particularly, computer vision and artificial intelligence are being recently used to estimate relevant parameters in agriculture and viticulture. The main goal of this PhD Thesis is to provide new tools in digital viticulture that combine computer vision and artificial intelligence towards yield components’ estimation under field conditions. In particular, the following objectives were attempted: i) the application of computer vision and machine learning for grapevine cluster compactness estimation; ii) the combination of deep learning and computer vision to quantify the number of grapevine flowers per vine; iii) the use of computer vision to analyze the impact of the canopy status on grapevine berry occlusion; iv) the combination of computer vision, deep learning and machine learning to quantify the number of grapevine pea-size berries per vine; and v) the use of computer vision, deep learning and machine learning to estimate grape yield months before harvest. For the first objective, RGB vine images were acquired on-the-go using a mobile sensing platform to obtain a compactness estimation of the grapevine clusters. A computer vision algorithm was developed to extract cluster morphology features from the images, and a machine learning model used those features to assess the compactness of each cluster. The results shown that the developed method can be a more objective alternative to traditional visual assessment performed by trained evaluators. For the second objective, which is focused on the quantification of grapevine flowers, a set of RGB vine images was acquired on-the-go at pre-flowering phenological stage using a mobile sensing platform. A deep learning semantic segmentation approach was followed to individually segment and count each flower presented on the images. A high correlation was found between the number of estimated flowers (from the images) and the final yield at harvest, proving that the developed system is highly useful to obtain a yield indicator near 100 days before harvest. The analysis of the impact of the canopy status on grapevine berry occlusion was addressed in a work where RGB vine images were manually acquired near harvest. From the results obtained it could be concluded that computer vision can be employed to assess yield on fully and partially defoliated vines in the fruiting zone, combined with a model capable to capture the variability in the canopy status from different vineyards. For the quantification of grapevine pea-size berries (fourth objective), a mobile sensing platform was employed to capture RGB vine images on-the-go at pea-size phenological stage. Deep learning semantic segmentation techniques were applied to obtain the number of visible berries in the images and several vine canopy features that were used by machine learning models to estimate the number of berries per vine, thus partially overcoming canopy occlusion artifacts. The results presented in this chapter demonstrated that this tool could be highly beneficial to develop a yield indicator almost two months before harvest without applying an intensive full vine defoliation. Finally, the grape yield assessment issue was presented. The images and vines from the work of the third objective were used. Computer vision, deep learning and machine learning were combined to obtain some canopy features, that were relevant to estimate the final yield overcoming canopy occlusions, in different grapevine varieties. The estimation model proved to be accurate at estimating the yield not only in grapevine varieties already included in the model, but also in new varieties not included. The outcomes presented in the research work of this PhD Thesis manifest the usefulness and applicability of computer vision, deep learning and machine learning to estimate grapevine yield components, non-invasively, under field conditions. These outcomes can be crucial in digital viticulture as an alternative to traditional methods, and a support for decision making in vineyards.