Degenerate Near-planar Road Surface 3D Reconstruction and Automatic Defects Detection

This dissertation presents an approach to reconstruct degenerate near-planar road surface in three-dimensional (3D) while automatically detect road defects. Three techniques are developed in this dissertation to establish the proposed approach. The first technique is proposed to reconstruct the degenerate near-planar road surface into 3D from one camera. Unlike the traditional Structure from Motion (SfM) technique which has the degeneracy issue for near-planar object 3D reconstruction, the uniqueness of the proposed technique lies in the use of near-planar characteristics of surfaces in the 3D reconstruction process, which solves the degenerate road surface reconstruction problem using only two images. Following the accuracy-enhanced 3D reconstructed road surface, the second technique automatically detects and estimates road surface defects. As the 3D surface is inversely solved from 2D road images, the detection is achieved by jointly identifying irregularities from the 3D road surfaces and the corresponding image information, while clustering road defects and obstacles using a mean-shift algorithm with flat kernel to estimate the depth, size, and location of the defects. To enhance the physics-driven automatic detection reliability, the third technique proposes and incorporates a self-supervised learning structure with data-driven Convolutional Neural Networks (CNN). Different from supervised learning approaches which need labeled training images, the road anomaly detection network is trained by road surface images that are automatically labeled based on the reconstructed 3D surface information. In order to collect clear road surface images on the public road, a road surface monitoring system is designed and integrated for the road surface image capturing and visualization. The proposed approach is evaluated in both simulated environment and through real-world experiments. The parametric study of the proposed approach shows the small error of the 3D road surface reconstruction influenced by different variables such as the image noise, camera orientation, and the vertical movement of the camera in a controlled simulation environment. The comparison with traditional SfM technique and the numerical results of the proposed reconstruction using real-world road surface images then indicate that the proposed approach effectively reconstructs high quality near-planar road surface while automatically detects road defects with high precision, accuracy, and recall rates without the degenerate issue.