To make this happen, two elements are essential! A small and lightweight camera that fits the platform and a powerful processing solution that turns the data into useful products. Ideally, hardware and software are developed in tandem, so they can be optimized together to generate high quality results. That is why we developed the spatio-spectral imager, a very innovative spectral imaging concept to build a miniature camera. It combines strong points of push-broom and frame based imagers and is ideally suited to capture images with both high spatial and spectral resolution. As a result, it collects more information per gram camera weight than competing systems.
Designing a miniature hyperspectral imaging system
The core of the camera is a frame sensor with a large number of hyperspectral filters deposited on the sensor pixels, with different spectral bands arranged in lines. A raw image taken with such a sensor looks a bit odd: it resembles a regular 2D image, but in fact every few lines it captures a different part of the electromagnetic spectrum.
Raw image as captured by a spatiospectral imager, measuring different wavelengths along the vertical axis
The challenges of collecting dataTo collect full spectra at each spot of the terrain, images are captured at high speed. Conventional frame cameras on drones typically acquire images at much lower speed, e.g. 1 per second. To avoid motion blur, the integration times are much shorter, e.g. 1/1000 second, so the camera is only collecting light for a very small fraction of the total flying time. By acquiring images much more rapidly, a spatiospectral camera can extract much more information in the same time.
The dense image sequences looks like an aerial movie which gradually explores an area. However, with every new image, the spatiospectral camera does not just capture new locations, but probes additional spectral information for all points in sight. A beautiful concept, but also a big challenge on the image processing side since the spectral information of one point on the ground is coded in multiple raw images.
Processing overview from left to right: matching locations in raw images, sorting measurements into planes per wavelength, retrieving spectra per point
Solving the spatiospectral puzzleSo how do we unravel the spectral information and produce high quality image products? We had to rethink the entire processing flow to solve this. Based on solid photogrammetric principles to ensure geometrically accurate results, we developed a unique and innovative set of methods containing dedicated tools to handle dense spatiospectral image sequences. The processing includes:
- aerial triangulation
- bundle block adjustment and outlier detection
- camera calibration
- dense point cloud extraction
- detailed 3D digital surface model generation
Camera systems generating useful informationCurrently, we are developing several spatiospectral cameras which benefit from this processing. The first result is the ButterflEYE LS camera, developed together with German camera builder Cubert gmbh. For this camera, the processing creates hypercubes with orthomosaiced hyperspectral reflectance maps.
For all our spatiospectral cameras, the goal is to provide a complete system which enables the users to concentrate on carrying out their flights, and be assured that the processing system takes care of the spectral data. The magic performed under the hood transforms this data into detailed useful information.
True color (left) and false color image (right) of the same area with complementary information