The Challenge of Hyperspectral Data
Hyperspectral data capture faces a significant challenge due to the gap between the volume of data produced and the limited downlink capacity. For example, the upcoming CSIMBA hyperspectral sensor on the IPERLITE mission can generate more than 5 TB of data each day. However, the daily downlink capacity is capped at under 20 GB, creating a severe bottleneck in data transfer.
This disparity underscores the critical need for advanced data compression solutions that can reduce data volume at the source, ensuring that essential information can be transmitted within existing bandwidth limits.
A general-purpose model for remote sensing
The CORSA model was initially developed as a part of the ESA PhiLab EO Science for Society initiative and has been further tuned ever since. It is a versatile neural network designed for near-lossless compression of satellite data.
The CORSA models, built on a multi-level quantized variational auto-encoder architecture, have shown their capability to compress hyperspectral and multispectral data, including Sentinel-1 and 2, PRISMA and APEX, by orders of magnitude.
Here are three standout features of the CORSA models:
1. High Compression Capability: CORSA models leverage a quantized variational auto-encoder to convert data into compact, lower-dimensional embeddings that retain all the essential details of the original. By quantizing this embedding space, only references to these embeddings need to be stored, achieving exceptionally high compression rates. For instance, Sentinel-2 images can be compressed up to 30 times, with potentially even higher ratios based on quality settings. In the case of hyperspectral data, such as ENMAP, the high correlation across spectral band enables compression rates as high as 300x!
2. Support for Downstream Applications: Beyond compression, CORSA models act as foundation models for a wide range of applications in both multispectral and hyperspectral analysis. CORSA embeddings can be seamlessly integrated into AI workflows for tasks like land cover classification or change detection. This adaptability also supports onboard implementations, enabling rapid disaster response and positioning CORSA as a versatile solution for diverse EO needs.
3. Versatility and Efficiency: CORSA models offer near-lossless compression for various data types, from multispectral to hyperspectral, and across different spatial resolutions. Designed with multiple architecture options, including convolutional and vision transformer backbones, each version has unique strengths. Despite these powerful capabilities, CORSA models can remain lightweight, making them suitable for deployment on edge devices and satellite platforms.
Use of CORSA embeddings for downstream tasks: Change Detection
Use of CORSA embeddings for downstream tasks: Landcover classification
Bringing CORSA to Nvidia Jetson:
Paving the way for near real-time onboard hyperspectral processing in satellites
The recent deployment of the CORSA data compression model on Nvidia's Jetson Orin NX hardware marks a significant step forward in enabling onboard hyperspectral data processing for satellite missions. The Jetson platform was chosen for its advanced System-on-Module (SoM) capabilities, which include a GA10B Ampere architecture GPU, multiple ARM Cortex CPUs, and a suite of CUDA and Tensor cores, all tailored for edge AI applications
Originally developed in PyTorch for the EnMAP satellite mission, the CORSA model was converted to the ONNX format and further optimized using TensorRT to maximize performance on the Jetson platform. This deployment was finely tuned for both nominal (12W) and high-power (25W) configurations, achieving exceptional efficiency in processing 128x128 pixel images with 54 spectral channels.
The evaluation of CORSA on the Jetson Orin NX showed impressive results in throughput, power consumption, and normalized efficiency (measured in MP/s/W), with the model consistently delivering high-quality image reconstruction across various power modes. Quality metrics - including PSNR, SAM, and SSIM - confirmed the model's robustness in maintaining detailed image fidelity, making it ideal for near real-time hyperspectral compression and onboard data analysis.
This optimized setup represents a significant advancement for satellite missions, as it minimizes the need for extensive data downlink by enabling immediate, onboard processing and analysis, allowing for fast, actionable insights from hyperspectral data. Future enhancements will explore Jetson's Deep Learning Accelerator (DLA) and apply quantization techniques to push efficiency even further. Additionally, efforts are underway to port the CORSA algorithm to the Hailo-8 AI accelerator, promising even greater performance gains for onboard satellite data processing.
Try it out now!
It's time to explore CORSA yourself now! We've developed a demo on the Terrascope platform where you can experiment with compressing Sentinel-2 images and examining the reconstructed outputs. You can even immediately use the compressed images to create landcover maps!
For the full experience, go to the CORSA GitHub to run the demo and investigate the results.
More details can be found
1. in the papers presented at IGARSS 2024 and OBPDC24:
- On-Board Hyperspectral Image Compression Using Vector-Quantized Auto Encoders
- Lightweight and Efficient: A family of Multimodal Earth Observation Foundation Models
- Implementation of CORSA on Nvidia Jetson Hardware
2. in projects:
3. at the CORSA webpage where you can also find our contact form!
