New CropSAR service provides Cloud-Free Time Series at Pixel Level

By Kristof Van Tricht 16 October 2023
In 2019 we released CropSAR, a service that combines both optical data from Sentinel-2 and radar data from Sentinel-1 to be able to look through the clouds and improve crop monitoring. CropSAR is still a valuable service to provide insights into crop performance, empowering farmers to make informed decisions for maximizing yields and sustainability.

Now, we’ve taken this technology to a higher level by upscaling CropSAR to generate cloud-free image time series at pixel level. We hereby not only improve the technology but also expand the capabilities and broaden its use. This new CropSAR service takes monitoring applications to new heights by extending to image level analysis, RGB insights and application domains beyond agriculture. Kristof Van Tricht explains more and shows how to get started.

Upscaling CropSAR to generate cloud-free time series at pixel level 

In 2019 we released CropSAR, a groundbreaking technology that combines Sentinel-1 radar and Sentinel-2 optical satellite data to generate cloud-free time series of field-based biophysical parameters such as NDVI. CropSAR provides valuable insights into the crop dynamics over time and contributes to both downstream applications as well as individual farmers through platforms such as WatchITgrow. Shortly after the release of CropSAR, we saw the potential to upscale the CropSAR technology from field level to pixel level so that we could generate cloud-free time series and analyse an entire region in detail rather than focus on one field.

We therefore developed an additional CropSAR service (CropSAR 2D) which allows us to produce cloud-free image time series at 10 m resolution every 5 days. This new service can take multiple monitoring application to the next level whereas the original CropSAR at field level (CropSAR 1D) focused specifically on agricultural applications.  
The animation below compares the monitoring of an agricultural area in Belgium during the 2020 growing season. The left panels are exclusively based on the original Sentinel-2 image time series, showing frequent gaps due to cloud cover. The right panels show the CropSAR predictions where cloud-induced gaps are filled with the help of Sentinel-1. The time series of two fields are shown in the bottom panels, demonstrating the improved consistency of the time series for monitoring applications using the newest CropSAR technology.

cropsar-px

Comparison of native Sentinel-2 with frequent cloud cover (left panels) and CropSAR generated equivalent without clouds (right panels).

Powerful gap-filling technology

To improve our CropSAR technology we teamed up with Superlinear, a company specialized in providing AI solutions, to combine state-of-the-art AI technologies with Earth observation domain expertise. Working at pixel level required an entirely new approach compared to the time series version of CropSAR. The new CropSAR service decouples the spatial and temporal dimensions. We use ResNet blocks for encoding and decoding individual images and a transformer encoder that explicitly focuses on the temporal dimension to perform the actual gap filling. During our experiments it became clear that this explicit decoupling enhances the gap-filling performance with limited artefacts in space and time.

Architecture

Conceptual CropSAR architecture decoupling spatial and temporal encoding and decoding for optimal reconstruction of clouded images.

Next to the focus on biophysical parameters such as NDVI, FAPAR and FCOVER, the additional CropSAR service can also generate cloud-free RGB-NIR image time series. This extension allows to monitor Earth’s surface using visual information as well, as shown below.

RGB_1

RGB_2

RGB image time series as originally observed by Sentinel-2 (left panels, cloud-masked) and as reconstructed by CropSAR (right panels, gap-filled).

one algorithm to serve many applications

The new CropSAR service can be used in many applications. Contrary to the original CropSAR service, which remains available as well, our new model was not trained exclusively on agricultural parcels which extends the use. Within the ESA AI4FOOD project, for example, CropSAR's performance was evaluated for the detection and monitoring of deforestation events in Uganda (see figure below). The new CropSAR service allowed us to detect deforestation and follow up pulses of regrowth of grass and shrubland during two wet seasons, typically characterized by prolonged periods of cloudiness. In an additional experiment of synthetic clouds uninterrupted for 5 months, CropSAR was still able to reconstruct the NDVI profile that allowed to detect the deforestation event.

AI4FOOD_usecase

NDVI profile generated by CropSAR covering a deforestation event in Uganda. The yellow curves are CropSAR predictions guided by a Sen2Cor SCL extended mask, while the red curve is a CropSAR prediction guided by a more aggressive UNET-based cloud mask. The lower panel represents an experiment for which a 5-month synthetic cloud gap was introduced (light blue) during which CropSAR still captures the general NDVI trend. Courtesy of AI4FOOD project.

Open access via Terrascope EO plaza

Both CropSAR services are available via Terrascope’s EOplaza, where they use OpenEO under the hood for highly scalable processing with limited user effort. You can launch a CropSAR request with just few lines of code and an internet connection.

While two CropSAR services are already available, we keep advancing the CropSAR technology. We aim to enhance the image sharpness in CropSAR-generated time series and we are also working on refining the cloud and shadow mask, which is crucial for guiding CropSAR's gap-filling process. Any inaccuracies in this mask, such as the sudden drop in the yellow curve in the NDVI profile above which is caused by an undetected cloud, have a direct impact on the CropSAR output. We already performed some experiments with an extended cloud mask based on a UNET (see the red curve) which demonstrated that this can eliminate the observed artefact.

As we continue to improve the CropSAR technology, we will conduct rigorous validation efforts to assess its performance under various conditions. Meanwhile, we invite any interested users to try out the new CropSAR (2D) service. We also encourage you to share your feedback via the dedicated CropSAR topic on the Terrascope forum so that we can provide the services you need.

Like this article? Share it on
Kristof Van Tricht
An article by
Kristof Van Tricht
Senior R&D Professional
More info about Kristof Van Tricht
Share

Related posts

New CropSAR service provides Cloud-Free Time Series at Pixel Level
  • Time series ,
  • Sentinel-2 ,
  • Sentinel-1 ,
  • CropSAR

New CropSAR service provides Cloud-Free Time Series at Pixel Level

By Kristof Van Tricht 16.10.2023
In 2019 we released CropSAR, a service that combines both optical data from Sentinel-2 and radar data from Sentinel-1 to be able to look through the..
Lees meer
Simplify your processing with the WorldCover annual composites
  • Sentinel-2 ,
  • land cover ,
  • global land cover ,
  • Sentinel-1 ,
  • worldcover

Simplify your processing with the WorldCover annual composites

By Daniele Zanaga 21.06.2023

During the development of ESA WorldCover 2020 and 2021, the first global land cover maps at 10 meters resolution based on Sentinel-1 and Sentinel-2, we..

Lees meer
Large scale mapping for biodiversity monitoring
  • Biodiversity ,
  • Sentinel-2 ,
  • global land cover ,
  • vegetation ,
  • GlobDiversity

Large scale mapping for biodiversity monitoring

By Ruben Van De Kerchove 09.02.2021

Last year, the European Space Agency finalized the GlobDiversity project, the first large-scale effort set up to develop and engineer remote sensing-based..

Lees meer