Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data

by   Barriere Valentin, et al.

Accurate in-season crop type classification is crucial for the crop production estimation and monitoring of agricultural parcels. However, the complexity of the plant growth patterns and their spatio-temporal variability present significant challenges. While current deep learning-based methods show promise in crop type classification from single- and multi-modal time series, most existing methods rely on a single modality, such as satellite optical remote sensing data or crop rotation patterns. We propose a novel approach to fuse multimodal information into a model for improved accuracy and robustness across multiple years and countries. The approach relies on three modalities used: remote sensing time series from Sentinel-2 and Landsat 8 observations, parcel crop rotation and local crop distribution. To evaluate our approach, we release a new annotated dataset of 7.4 million agricultural parcels in France and Netherlands. We associate each parcel with time-series of surface reflectance (Red and NIR) and biophysical variables (LAI, FAPAR). Additionally, we propose a new approach to automatically aggregate crop types into a hierarchical class structure for meaningful model evaluation and a novel data-augmentation technique for early-season classification. Performance of the multimodal approach was assessed at different aggregation level in the semantic domain spanning from 151 to 8 crop types or groups. It resulted in accuracy ranging from 91% to 95% for NL dataset and from 85% to 89% for FR dataset. Pre-training on a dataset improves domain adaptation between countries, allowing for cross-domain zero-shot learning, and robustness of the performances in a few-shot setting from France to Netherlands. Our proposed approach outperforms comparable methods by enabling learning methods to use the often overlooked spatio-temporal context of parcels, resulting in increased preci...


page 4

page 10

page 14

page 15

page 16

page 17


Multimodal Crop Type Classification Fusing Multi-Spectral Satellite Time Series with Farmers Crop Rotations and Local Crop Distribution

Accurate, detailed, and timely crop type mapping is a very valuable info...

Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series

Optical and radar satellite time series are synergetic: optical images c...

Semantic Segmentation of Vegetation in Remote Sensing Imagery Using Deep Learning

In recent years, the geospatial industry has been developing at a steady...

Towards Global Crop Maps with Transfer Learning

The continuous increase in global population and the impact of climate c...

Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series

While annual crop rotations play a crucial role for agricultural optimiz...

Sentinel-1 and Sentinel-2 Spatio-Temporal Data Fusion for Clouds Removal

The abundance of clouds, located both spatially and temporally, often ma...

Multi-Modal Prototype Learning for Interpretable Multivariable Time Series Classification

Multivariable time series classification problems are increasing in prev...

Please sign up or login with your details

Forgot password? Click here to reset