Development of Ground-Level Hyperspectral Image Datasets and Analysis Tools, and their use towards a Feature Selection based Sensor Design Method for Material Classification

Abstract

Visual sensing in robotics, especially in the context of autonomous vehicles, has advanced quickly and many important contributions have been made in the areas of target classification. Typical to these studies is the use of the Red-Green-Blue (RGB) camera. Separately, in the field of remote sensing, the hyperspectral camera has been used to perform classification tasks on natural and man-made objects from typically aerial or satellite platforms. Hyperspectral data is characterized by a very fine spectral resolution, resulting in a significant increase in the ability to identify materials in the image. This hardware has not been studied in the context of autonomy as the sensors are large, expensive, and have non-trivial image capture times.

This work presents three novel contributions: a Labeled Hyperspectral Image Dataset (LHID) of ground-level, outdoor objects based on typical scenes that a vehicle or pedestrian may encounter, an open-source hyperspectral interface software package (HSImage), and a feature selection based sensor design algorithm for object detection sensors (DLSD). These three contributions are novel and useful in the fields of hyperspectral data analysis, visual sensor design, and hyperspectral machine learning. The hyperspectral dataset and hyperspectral interface software were used in the design and testing of the sensor design algorithm.

The LHID is shown to be useful for machine learning tasks through experimentation and provides a unique data source for hyperspectral machine learning. HSImage is shown to be useful for manipulating, labeling and interacting with hyperspectral data, and allows wavelength and classification based data retrieval, storage of labeling information and ambient light data. DLSD is shown to be useful for creating wavelength bands for a sensor design that increase the accuracy of classifiers trained on data from the LHID. DLSD shows accuracy near that of the full spectrum hyperspectral data, with a reduction in features on the order of 100 times. It compared favorably to other state-of-the-art wavelength feature selection techniques and exceeded the accuracy of an RGB sensor by 10%.