Upscaling CropSAR to generate cloud-free time series at pixel level
In 2019 we released CropSAR, a groundbreaking technology that combines Sentinel-1 radar and Sentinel-2 optical satellite data to generate cloud-free time series of field-based biophysical parameters such as NDVI. CropSAR provides valuable insights into the crop dynamics over time and contributes to both downstream applications as well as individual farmers through platforms such as WatchITgrow. Shortly after the release of CropSAR, we saw the potential to upscale the CropSAR technology from field level to pixel level so that we could generate cloud-free time series and analyse an entire region in detail rather than focus on one field.
We therefore developed an additional CropSAR service (CropSAR 2D) which allows us to produce cloud-free image time series at 10 m resolution every 5 days. This new service can take multiple monitoring application to the next level whereas the original CropSAR at field level (CropSAR 1D) focused specifically on agricultural applications.
The animation below compares the monitoring of an agricultural area in Belgium during the 2020 growing season. The left panels are exclusively based on the original Sentinel-2 image time series, showing frequent gaps due to cloud cover. The right panels show the CropSAR predictions where cloud-induced gaps are filled with the help of Sentinel-1. The time series of two fields are shown in the bottom panels, demonstrating the improved consistency of the time series for monitoring applications using the newest CropSAR technology.
Comparison of native Sentinel-2 with frequent cloud cover (left panels) and CropSAR generated equivalent without clouds (right panels).
Powerful gap-filling technology
To improve our CropSAR technology we teamed up with Radix, a company specialized in providing AI solutions, to combine state-of-the-art AI technologies with Earth observation domain expertise. Working at pixel level required an entirely new approach compared to the time series version of CropSAR. The new CropSAR service decouples the spatial and temporal dimensions. We use ResNet blocks for encoding and decoding individual images and a transformer encoder that explicitly focuses on the temporal dimension to perform the actual gap filling. During our experiments it became clear that this explicit decoupling enhances the gap-filling performance with limited artefacts in space and time.
Conceptual CropSAR architecture decoupling spatial and temporal encoding and decoding for optimal reconstruction of clouded images.
Next to the focus on biophysical parameters such as NDVI, FAPAR and FCOVER, the additional CropSAR service can also generate cloud-free RGB-NIR image time series. This extension allows to monitor Earth’s surface using visual information as well, as shown below.
RGB image time series as originally observed by Sentinel-2 (left panels, cloud-masked) and as reconstructed by CropSAR (right panels, gap-filled).
one algorithm to serve many applications
The new CropSAR service can be used in many applications. Contrary to the original CropSAR service, which remains available as well, our new model was not trained exclusively on agricultural parcels which extends the use. Within the ESA AI4FOOD project, for example, CropSAR's performance was evaluated for the detection and monitoring of deforestation events in Uganda (see figure below). The new CropSAR service allowed us to detect deforestation and follow up pulses of regrowth of grass and shrubland during two wet seasons, typically characterized by prolonged periods of cloudiness. In an additional experiment of synthetic clouds uninterrupted for 5 months, CropSAR was still able to reconstruct the NDVI profile that allowed to detect the deforestation event.
NDVI profile generated by CropSAR covering a deforestation event in Uganda. The yellow curves are CropSAR predictions guided by a Sen2Cor SCL extended mask, while the red curve is a CropSAR prediction guided by a more aggressive UNET-based cloud mask. The lower panel represents an experiment for which a 5-month synthetic cloud gap was introduced (light blue) during which CropSAR still captures the general NDVI trend. Courtesy of AI4FOOD project.
Open access via Terrascope EO plaza
Both CropSAR services are available via Terrascope’s EOplaza, where they use OpenEO under the hood for highly scalable processing with limited user effort. You can launch a CropSAR request with just few lines of code and an internet connection.
While two CropSAR services are already available, we keep advancing the CropSAR technology. We aim to enhance the image sharpness in CropSAR-generated time series and we are also working on refining the cloud and shadow mask, which is crucial for guiding CropSAR's gap-filling process. Any inaccuracies in this mask, such as the sudden drop in the yellow curve in the NDVI profile above which is caused by an undetected cloud, have a direct impact on the CropSAR output. We already performed some experiments with an extended cloud mask based on a UNET (see the red curve) which demonstrated that this can eliminate the observed artefact.
As we continue to improve the CropSAR technology, we will conduct rigorous validation efforts to assess its performance under various conditions. Meanwhile, we invite any interested users to try out the new CropSAR (2D) service. We also encourage you to share your feedback via the dedicated CropSAR topic on the Terrascope forum so that we can provide the services you need.