Real-time Risk Prediction Using Temporal Gaze Pattern and Evidence Accumulation

Driver gaze information provides insight into the driver’s perception and decision-making process in a dynamic driving environment. It is important to learn whether driver gaze information immediately before a safety-critical event (SCE) can be used to assess crash risk, and if so, how early in the crash we can predict the potential crash risk. This requires a thorough understanding of the temporal pattern of the data. In this project, we explored multiple key research questions pertaining to driver gaze and its relation to SCEs. We first looked at how temporal gaze patterns in SCEs like crashes and near-crashes are different from baseline events. Then we extended this analysis to understand if the temporal gaze pattern can indicate the direction of the threat. Finally, we showed how contextual information can aid the understanding of gaze patterns. We used traffic density, positions of traffic actors, and vehicle speed as key factors that can influence the temporal nature of gaze and impact safety. This study introduces a real-time crash risk prediction framework leveraging deep learning models based on driver gaze and contextual data. The analysis utilizes the Second Strategic Highway Research Program (SHRP 2) Naturalistic Driving Study, preprocessing it to extract temporal driver gaze patterns and contextual data for two distinct subsets of data. Dataset 1 comprises one-dimensional temporal driver gaze patterns only (named “SHRP2_Gaze”). Dataset 2 includes additional contextual information such as vehicle speed, distance to lead vehicle, and level of service (named “SHRP2_Gaze_Context).” Dataset 1 was used to explore the feasibility of predicting crash risk solely based on driver gaze data. Dataset 2 was applied to assessing the potential for early crash warnings by analyzing both driver gaze patterns and contextual data. For SHRP2_Gaze, four deep learning models (long short-term memory [LSTM], one-dimensional convolutional neural network [1D CNN], 1D CNN+LSTM, and 1D CNN+LSTM+Attention) were trained and tested on various inputs (Input 1: 20 seconds from five zones; Input 2: different lengths from five zones; and Input 3: 20 seconds from 19 zones), considering different lengths and zones of driver gaze data. The 1D CNN with XGBoost exhibited superior performance in crash risk prediction but demonstrated challenges in distinguishing specific crash and near-crash events. This portion of the study emphasized the significance of the temporal gaze pattern length and zone selection for real-time crash risk prediction using driver gaze data only. With SHRP2_Gaze_Context, the 1D CNN model with XGBoost emerged as the top-performing model when utilizing 11 types of 20-second driver gaze and contextual data to predict crash, near-crash, and baseline events. Notably, vehicle speed proves highly correlated with event categories, showcasing the effectiveness of combining driver gaze and contextual data. The study also highlighted the model’s ability to adapt to early crash warning scenarios, leveraging different 2-second multivariate data without an increase in computing time. The deep learning models employed in this study outperform traditional statistical models in distinguishing features from driver gaze and contextual data. These findings hold significant implications for accurate and efficient real-time crash risk prediction and early crash warning systems. The study’s insights cater to public agencies, insurance companies, and fleets, providing valuable understanding of driver behavior and contextual information leading to diverse safety events.