Spatiospectral processing, the key to high resolution hyperspectral imaging

Designing a miniature hyperspectral imaging system

To make this happen, two elements are essential! A small and lightweight camera that fits the platform and a powerful processing solution that turns the data into useful products. Ideally, hardware and software are developed in tandem, so they can be optimized together to generate high quality results. That is why we developed the spatio-spectral imager, a very innovative spectral imaging concept to build a miniature camera. It combines strong points of push-broom and frame based imagers and is ideally suited to capture images with both high spatial and spectral resolution. As a result, it collects more information per gram camera weight than competing systems.

The core of the camera is a frame sensor with a large number of hyperspectral filters deposited on the sensor pixels, with different spectral bands arranged in lines. A raw image taken with such a sensor looks a bit odd: it resembles a regular 2D image, but in fact every few lines it captures a different part of the electromagnetic spectrum.

Raw image as captured by a spatiospectral imager, measuring different wavelengths along the vertical axis

The challenges of collecting data

To collect full spectra at each spot of the terrain, images are captured at high speed. Conventional frame cameras on drones typically acquire images at much lower speed, e.g. 1 per second. To avoid motion blur, the integration times are much shorter, e.g. 1/1000 second, so the camera is only collecting light for a very small fraction of the total flying time. By acquiring images much more rapidly, a spatiospectral camera can extract much more information in the same time.

The dense image sequences looks like an aerial movie which gradually explores an area. However, with every new image, the spatiospectral camera does not just capture new locations, but probes additional spectral information for all points in sight. A beautiful concept, but also a big challenge on the image processing side since the spectral information of one point on the ground is coded in multiple raw images.

Processing overview from left to right: matching locations in raw images, sorting measurements into planes per wavelength, retrieving spectra per point

Solving the spatiospectral puzzle

So how do we unravel the spectral information and produce high quality image products? We had to rethink the entire processing flow to solve this. Based on solid photogrammetric principles to ensure geometrically accurate results, we developed a unique and innovative set of methods containing dedicated tools to handle dense spatiospectral image sequences. The processing includes:

aerial triangulation
bundle block adjustment and outlier detection
camera calibration
dense point cloud extraction
detailed 3D digital surface model generation

The spectral information at all wavelengths is extracted from the input images and is projected onto hyperspectral layers so it becomes aligned at pixel level. Radiometric and spectral corrections are applied to generate reliable spectral reflectance products, from which maps containing useful information such as vegetation indices and biophysical parameter estimates are derived for precision agriculture and environmental monitoring. All this bundled in a generic automated processing solution which can be adapted to various camera systems.

Camera systems generating useful information

Currently, we are developing several spatiospectral cameras which benefit from this processing. The first result is the ButterflEYE LS camera, developed together with German camera builder Cubert gmbh. For this camera, the processing creates hypercubes with orthomosaiced hyperspectral reflectance maps.

For all our spatiospectral cameras, the goal is to provide a complete system which enables the users to concentrate on carrying out their flights, and be assured that the processing system takes care of the spectral data. The magic performed under the hood transforms this data into detailed useful information.