SPECIFIC AIMS:

This project aims to develop a data-efficient, AI-enabled framework for water (W) and nitrogen (N) stress sensing and prediction. To do this, we will pursue the following sub-objectives: (Obj. 1) Design, build, and deploy W and N status sensors that will accelerate the rate of ground-truth data collection, (Obj. 2) Develop a synthetic data generation pipeline that couples process-based biophysical modeling to deep learning for prediction of W and N stress, and (Obj. 3) Create a deep learning framework that fuses multiple data streams to directly predict plant W and N stress at multiple scales from individual tree to orchard (Obj. 1 and 2). We will pursue these objectives for almond, which is the most widely irrigated and economically valuable crop in California.

BACKGROUND:

Direct methods for W and N stress measurement are expensive, labor-intensive, and difficult to upscale. During the last few decades, substantial research has focused on developing indirect low cost in-situ, proximal, and remote sensing systems for near real-time detection of W and N stress. Most of these sensing systems measure a surrogate property e.g., soil dielectric permittivity, leaf temperature, leaf/canopy chlorophyll content, and evapotranspiration-based water deficit. These indirect methods tend to be associated with high uncertainty due to sensor and measurement error. The problem is compounded in crops such as almonds with complex root systems. There is no standard and scalable technology available for predicting W and N stress in a cost-effective and timely way. We seek to develop a robust framework that integrates sensing, biophysical modeling, and artificial intelligence to predict W and N stress.

Several key obstacles limit the widespread adoption of these technologies for sustainable agricultural production. First, no large scale, high-resolution ground-truth W, and N stress datasets exist that encompass the broad range of environmental conditions, crop traits, and management strategies present across specialty crops. Not only does this prevent validation of how accurately any sensing systems detect stress but also how well biophysical models predict W and N stress. It also strongly limits our ability to take full advantage of state-of-the-art deep learning models designed to discover and understand complex patterns from very large datasets.

Second, even if we successfully accelerate ground-truth W and N stress data collection, the task of doing this in a way that covers the diversity of existing crop types, cultivars, environments, and management conditions will require massive effort and cost. Previous work from other fields, such as autonomous driving, has demonstrated the potential for pre-training machine learning models on synthetic data. While significant progress has been made in developing state-of-the-art biophysical models that simulate W & N stress, these models have not been used for synthetic data generation in agricultural crops. Given its potential for substantially increasing data-efficiency, there is a need to develop a synthetic data generation pipeline that couples process-based biophysical modeling to deep learning for W and N stress.

Previous work has shown that W and N stress are driven by a complex interaction between soil, weather, genetics and management. Therefore, to predict W and N stress, a large number of variables need to be known prior. These variables can be obtained from ground-based observations (e.g., soil W, soil and plant NO3 content, irrigation, fertigation, planting density, cultivar, tree size/age, and canopy coverage, pests and disease pressure, microclimate variables), and remotely-sensed variables (canopy structure, canopy greenness & temperature, etc.). Moreover, ground-based data provides a very different spatial perspective than aerial sensing. Despite the potentially unique and complementary information contained in ground-based and aerial imagery data, no deep learning modeling techniques exist that fuse these multiple data streams. There is a need to create a deep learning framework that fuses multiple data streams to predict plant W and N stress.

SIGNIFICANCE AND IMPACT:

Rapid detection and prediction of W and N stress is vital for optimizing production. Moreover, monitoring and predicting stresses at high spatio-temporal resolution enables precision management activities, mechanization, and automation that can increase grower output and quality, while minimizing negative environmental impacts. For example, early detection of W stress can identify potential irrigation problems. Despite its economic and environmental significance, W and N stress prediction are very challenging for specialty crops, due to their complex structure and physiology, diverse crop traits, and a wide range of environmental conditions and management strategies. By developing an AI-enabled framework for near real-time monitoring and a prediction that are generalizable across many specialty crops, we expect to transform US food systems by innovating AI technology for a more sustainable food production system.

APPROACH WITH SUCCESS METRICS:

[Obj. 1] To dramatically accelerate the rate of ground-truth W and N data collection, we will design, build, and deploy W and N stress sensors. Specifically, we will instrument existing commercial farms with in-situ, proximal, and remote sensing systems for measuring W and N status in soil and plants at multiple scales. Currently, Kisekka, Pourreza, and Bailey are working with commercial partners to deploy a wide range of sensors. Success metrics for Obj. 1 in Y1 are the generation of ground-truth W and N stress data.

[Obj. 2] To substantially increase data-efficiency and model generalizability, we will develop a synthetic data generation pipeline that couples biophysical modeling and deep learning for W and N stress prediction. A 3D crop model called Helios will generate millions of 2D synthetic sensor imagery (i.e. thermal and chlorophyll) of almond cultivars across combinations of environmental and management scenarios, with trees of estimated W and N stress values. We will then use this synthetic data to pre-train models to estimate W and N stress in actual production conditions. Success metrics for Obj. 2 in Y1 are the generation of synthetic training data for almond and a pre-trained model that is validated against real-world ground-truth data.

[Obj. 3] To overcome current models’ inability to predict W and N stress with high accuracy at individual tree scale, we will create a deep learning framework that fuses multiple ground-based and aerial data streams to directly predict W and N. We will build on current efforts by Kisekka, Jin and Pourreza to predict W and N stress in almonds. Novel deep learning architectures will be developed to fuse multiple data. Success metrics for Obj. 3 in Y1 are the curation of training datasets of ground-based and aerial sensor data with corresponding ground-truth W and N stress values, and an accompanying end-to-end deep learning training and evaluation pipeline.

Project Team

Collaboration

The projects are jointly conducted with partners through the AIFS Network:
https://aifs.ucdavis.edu/

UC Davis

UC Berkeley