Time Delay Mitigation in Aerial Telerobotic Operations Using Predictors and Predictive Displays

Semi-autonomous uncrewed aerial vehicles (UAVs) are telerobotic operations by definition where the UAV assumes the role of a telerobot and the human assumes the role of a supervisor. All telerobotic operations are susceptible to time delays due to communication, mechanical, and other constraints. Typically, these delays are small and do not affect the telerobotic operation for most of the tasks. However, for long-distance telerobotic operations like interplanetary rovers, deep underwater vehicles, etc. the delays can be so significant that they can render the entire operation void. This dissertation investigates the use of a novel heterogeneous stereo-vision system to mitigate the effects of time delays in a UAV-based visual interface presented to a human operator. The heterogeneous stereo-vision system consists of an omnidirectional camera and a pan-tilt-zoom camera. Two predictive display setups were developed that modify the delayed video imagery that would otherwise be presented to the operator in a way that provides an almost immediate visual response to the operator's control actions. The usability of the system is determined through human performance testing with and without the predictive algorithms. The results indicate that the predictive algorithm allows more efficient, accurate, and user-friendly operation. The second half of the dissertation deals with improving the performance of the predictive display and expanding the concept of the prediction from a stationary gimbal-camera system to a moving 6 DoF aircraft. Specifically, it talks about a novel extended Kalman filter (EKF)-based nonlinear predictor – the extended Kalman predictor (EKP) – and compares its performance with two linear predictors, the Smith predictor (SP) and the Kalman predictor (KP). This dissertation provides the mathematical formulation of the EKP, as well as the two linear predictors, and describes their use with simulated flight data obtained using a nonlinear motion model for a small, fixed-wing UAV. The EKP performs comparably to the KP when the aircraft motion experiences small perturbations from a nominal trajectory, but the EKP outperforms the KP for larger excursions. The SP performs poorly in every case.