Improved 2D Camera-Based Multi-Object Tracking for Autonomous Vehicles

Abstract

Effective multi-object tracking is crucial for autonomous vehicles to navigate safely and efficiently in dynamic environments. To make autonomous vehicles more affordable one area to address is the computational limitations of the sensors, therefore, cameras are often the first choice sensor. Three challenges in implementation of multi-object tracking in autonomous vehicles are: 1) In these vehicles, sensors like cameras are not static, which can cause motion blur in the frames and make tracking inefficient. 2) Traditional methods for motion compensation, such as those used in Kalman Filter-based Multi-Object Tracking, require extensive parameter tuning to match features between consecutive frames accurately. 3) Simple intersection over union (IoU) metric is insufficient for reliable identification in such environments. This thesis proposes a novel methodology for 2D multi-object tracking in autonomous vehicles using a camera-based Tracking-by-Detection (TBD) approach, emphasizing four key innovations: (1) A real-time deblurring module to mitigate motion blur, ensuring clearer frames for accurate detection; (2) deep learning-based motion compensation module that adapts dynamically to varying motion patterns, enhancing robustness; (3) adaptive cost function for association, incorporating object appearance and temporal consistency to improve upon traditional IoU metrics; (4) The integration of the Unscented Kalman Filter to effectively address non-linearities in the tracking process, enhancing state estimation accuracy. To maintain a Simple Online and Realtime (SORT) framework, we enhance detection by fine-tuning YOLOv8 and YOLOv9 models using autonomous driving datasets like BDD100K and KITTI, which are specifically tailored for these scenarios. Additionally, we incorporate a non-linear approach using the UKF to better capture the influence of various tracking dynamics, further improving tracking performance. Our evaluations show that the proposed methodology significantly outperforms existing state-of-the-art methods while maintaining the same inference rate as the baseline SORT model. These advancements not only improve the accuracy and reliability of multi-object tracking but also reduce the computational burden associated with parameter tuning and motion compensation. Consequently, this work presents a robust and efficient tracking solution for autonomous vehicles, making it viable for real-world deployment under both computational and cost constraints.