VTechWorks Repository :: Browsing by Author "Sarkar, Abhijit"

Browsing by Author "Sarkar, Abhijit"

Now showing 1 - 20 of 24

Analysis of Car Cut-ins Between Trucks Based on Existing Naturalistic Driving Data
Sarkar, Abhijit; Engström, Johan; Hanowski, Richard J. (National Surface Transportation Safety Center for Excellence, 2022-03-21)
For successful operation of cooperative adaptive cruise control, the participating vehicles follow certain operational criteria. For truck platooning, the participating trucks are required to maintain a minimum inter-vehicular distance for efficient vehicle-to-vehicle communication and better fuel efficiency. In order to ensure such operations, it is necessary to study the behavior of other agents in the roadway that may disrupt a platooning chain. A car cut-in is generally regarded as a disruption to the natural flow of traffic. In general, a cut-in takes place when a vehicle from the adjacent lane comes between the host vehicle and a lead vehicle, and in turn becomes the lead vehicle. In this work, we have searched 2.1 million miles of naturalistic truck driving data to identify candidate close cut-in scenarios. Then we analyzed approximately 18,500 cut-in cases to study the effects of car cut-ins under different platooning operating conditions, including following distance and headway. This study demonstrates the probability of cut-ins as a function of the following distance. The study found that the probability of cut-in increases when the host vehicle keeps a following distance greater than 23.5 meters. It also shows that as a result of a cut-in, the host vehicle often needs to brake and increase distance with the original lead vehicle by 15.5 meters. This scenario shows how the following vehicle reacts to a cut-in scenario. We further analyzed the safety implications of cut-in situations by computing the changes in time to collision. As the following behavior of the driver is one of the major factors governing cut-ins, we also analyzed the typical following behavior of the drivers in terms of following distance, following duration, following headway, and following speed. We assessed the safety of typical following distances through mathematical models. We believe that the study will significantly benefit researchers working with platoon control systems and coordinated platooning models to develop strategies towards the successful deployment of platooning and countermeasures for cut ins.
Assessment of Psychophysiological Characteristics of Drivers Using Heart Rate from SHRP2 Face Video Data
Sarkar, Abhijit; Doerzaph, Zachary R.; Abbott, A. Lynn (2014-08-25)
The goal is to

Extract heart rate from face video

Understand the behavior of driver, e.g. cognitive load, panic attack, drowsiness, DUI

Develop automatic video reduction technique

Devise a tool for future
Behavior-Based Predictive Safety Analytics Phase II
Miller, Andrew M.; Sarkar, Abhijit; McDonald, Tony; Ghanipoor-Machiani, Sahar; Jahangiri, Arash (Safe-D National UTC, 2023-06)
This project addressed the emerging field of behavior-based predictive safety analytics, focusing on the prediction of road crash involvement based on individual driver behavior characteristics. This has a range of applications in the areas of fleet safety management and insurance, but may also be used to evaluate the potential safety benefits of an automated driving system. This project continued work from a pilot study that created a proof-of-concept demonstration on how crash involvement may be predicted on the basis of individual driver behavior utilizing naturalistic data from the Second Strategic Highway Research Program. The current project largely focuses on understanding and identifying the risks from a driver based on their driving behaviors, personal characteristics, and environmental influences. This project analyzed large scale continuous naturalistic data as well as event data to study the role of different driving behaviors in the buildup of risk related to a safety-critical event or crash. This research can be used structure the development of real-time crash risk that accounts for those identified driver behaviors to be evaluated across the contextualized information on a roadway.
Camera-based Feature Identification for EasyMile Operation
Sarkar, Abhijit; Sundharam, Vaibhav; Manke, Aditi; Grove, Kevin (National Surface Transportation Safety Center for Excellence, 2022-11-15)
The EasyMile deployment studied in this work included cameras that captured the 360 degrees of roadway environment around the vehicle. We developed a scene perception algorithm using computer vision technology to track other roadway agents like cars, pedestrians, and bicyclists around the EasyMile LSAV. We used object detection and tracking algorithms to track the trajectories of each of the roadway agents. Then we used perspective geometry and camera specifications to find the relative distances and speeds of these agents with respect to the EasyMile. This helped us understand the configurations of the traffic around the LSAV and study other drivers’ temporal behavior. For example, the collected data shows the approach of any vehicle towards the EasyMile. Finally, we used this information to study other vehicles’ maneuvers and show how the information from the cameras can be used to study simple maneuvers of other vehicles such as cut-ins, lane changes, and following behavior. Through these camera-based tools, we have demonstrated examples from the real-world deployment. We studied following behavior characteristics that show the relative distance and speed of other vehicles’ following behavior. We have also demonstrated cut-in behaviors through the longitudinal and lateral trajectories of cut-in vehicles. We also showed how abrupt cut-ins may lead the EasyMile to apply its brakes, leading to safety critical events for following vehicles. Finally, we demonstrated how pedestrian behavior can be studied via these camera-based methods.
Camera-based Recovery of Cardiovascular Signals from Unconstrained Face Videos Using an Attention Network
Deshpande, Yogesh Rajan (Virginia Tech, 2023-06-22)
This work addresses the problem of recovering the morphology of blood volume pulse (BVP) information from a video of a person's face. Video-based remote plethysmography methods have shown promising results in estimating vital signs such as heart rate and breathing rate. However, recovering the instantaneous pulse rate signals is still a challenge for the community. This is due to the fact that most of the previous methods concentrate on capturing the temporal average of the cardiovascular signals. In contrast, we present an approach in which BVP signals are extracted with a focus on the recovery of the signal morphology as a generalized form for the computation of physiological metrics. We also place emphasis on allowing natural movements by the subject. Furthermore, our system is capable of extracting individual BVP instances with sufficient signal detail to facilitate candidate re-identification. These improvements have resulted in part from the incorporation of a robust skin-detection module into the overall imaging-based photoplethysmography (iPPG) framework. We present extensive experimental results using the challenging UBFC-Phys dataset and the well-known COHFACE dataset. The source code is available at https://github.com/yogeshd21/CVPM-2023-iPPG-Paper.
Cardiac Signals: Remote Measurement and Applications
Sarkar, Abhijit (Virginia Tech, 2017-08-25)
The dissertation investigates the promises and challenges for application of cardiac signals in biometrics and affective computing, and noninvasive measurement of cardiac signals. We have mainly discussed two major cardiac signals: electrocardiogram (ECG), and photoplethysmogram (PPG). ECG and PPG signals hold strong potential for biometric authentications and identifications. We have shown that by mapping each cardiac beat from time domain to an angular domain using a limit cycle, intra-class variability can be significantly minimized. This is in contrary to conventional time domain analysis. Our experiments with both ECG and PPG signal shows that the proposed method eliminates the effect of instantaneous heart rate on the shape morphology and improves authentication accuracy. For noninvasive measurement of PPG beats, we have developed a systematic algorithm to extract pulse rate from face video in diverse situations using video magnification. We have extracted signals from skin patches and then used frequency domain correlation to filter out non-cardiac signals. We have developed a novel entropy based method to automatically select skin patches from face. We report beat-to-beat accuracy of remote PPG (rPPG) in comparison to conventional average heart rate. The beat-to-beat accuracy is required for applications related to heart rate variability (HRV) and affective computing. The algorithm has been tested on two datasets, one with static illumination condition and the other with unrestricted ambient illumination condition. Automatic skin detection is an intermediate step for rPPG. Existing methods always depend on color information to detect human skin. We have developed a novel standalone skin detection method to show that it is not necessary to have color cues for skin detection. We have used LBP lacunarity based micro-textures features and a region growing algorithm to find skin pixels in an image. Our experiment shows that the proposed method is applicable universally to any image including near infra-red images. This finding helps to extend the domain of many application including rPPG. To the best of our knowledge, this is first such method that is independent of color cues.
Color Invariant Skin Segmentation
Xu, Han (Virginia Tech, 2022-03-25)
This work addresses the problem of automatically detecting human skin in images without reliance on color information. Unlike previous methods, we present a new approach that performs well in the absence of such information. A key aspect of the work is that color-space augmentation is applied strategically during the training, with the goal of reducing the influence of features that are based entirely on color and increasing more semantic understanding. The resulting system exhibits a dramatic improvement in performance for images in which color details are diminished. We have demonstrated the concept using the U-Net architecture, and experimental results show improvements in evaluations for all Fitzpatrick skin tones in the ECU dataset. We further tested the system with RFW dataset to show that the proposed method is consistent across different ethnicities and reduces bias to any skin tones. Therefore, this work has strong potential to aid in mitigating bias in automated systems that can be applied to many applications including surveillance and biometrics.
Color Invariant Skin Segmentation
Xu, Han; Sarkar, Abhijit; Abbott, A. Lynn (IEEE, 2022-06)
This paper addresses the problem of automatically detecting human skin in images without reliance on color information. A primary motivation of the work has been to achieve results that are consistent across the full range of skin tones, even while using a training dataset that is significantly biased toward lighter skin tones. Previous skin-detection methods have used color cues almost exclusively, and we present a new approach that performs well in the absence of such information. A key aspect of the work is dataset repair through augmentation that is applied strategically during training, with the goal of color invariant feature learning to enhance generalization. We have demonstrated the concept using two architectures, and experimental results show improvements in both precision and recall for most Fitzpatrick skin tones in the benchmark ECU dataset. We further tested the system with the RFW dataset to show that the proposed method performs much more consistently across different ethnicities, thereby reducing the chance of bias based on skin color. To demonstrate the effectiveness of our work, extensive experiments were performed on grayscale images as well as images obtained under unconstrained illumination and with artificial filters. Source code: https://github.com/HanXuMartin/Color-Invariant-Skin-Segmentation
Comprehensive Assessment of Artificial Intelligence Tools for Driver Monitoring and Analyzing Safety Critical Events in Vehicles
Yang, Guangwei; Ridgeway, Christie; Miller, Andrew M.; Sarkar, Abhijit (MDPI, 2024-04-12)
Human factors are a primary cause of vehicle accidents. Driver monitoring systems, utilizing a range of sensors and techniques, offer an effective method to monitor and alert drivers to minimize driver error and reduce risky driving behaviors, thus helping to avoid Safety Critical Events (SCEs) and enhance overall driving safety. Artificial Intelligence (AI) tools, in particular, have been widely investigated to improve the efficiency and accuracy of driver monitoring or analysis of SCEs. To better understand the state-of-the-art practices and potential directions for AI tools in this domain, this work is an inaugural attempt to consolidate AI-related tools from academic and industry perspectives. We include an extensive review of AI models and sensors used in driver gaze analysis, driver state monitoring, and analyzing SCEs. Furthermore, researchers identified essential AI tools, both in academia and industry, utilized for camera-based driver monitoring and SCE analysis, in the market. Recommendations for future research directions are presented based on the identified tools and the discrepancies between academia and industry in previous studies. This effort provides a valuable resource for researchers and practitioners seeking a deeper understanding of leveraging AI tools to minimize driver errors, avoid SCEs, and increase driving safety.
Deidentification of Face Videos in Naturalistic Driving Scenarios
Thapa, Surendrabikram (Virginia Tech, 2023-09-05)
The sharing of data has become integral to advancing scientific research, but it introduces challenges related to safeguarding personally identifiable information (PII). This thesis addresses the specific problem of sharing drivers' face videos for transportation research while ensuring privacy protection. To tackle this issue, we leverage recent advancements in generative adversarial networks (GANs) and demonstrate their effectiveness in deidentifying individuals by swapping their faces with those of others. Extensive experimentation is conducted using a large-scale dataset from ORNL, enabling the quantification of errors associated with head movements, mouth movements, eye movements, and other human factors cues. Additionally, qualitative analysis using metrics such as PERCLOS (Percentage of Eye Closure) and human evaluators provide valuable insights into the quality and fidelity of the deidentified videos. To enhance privacy preservation, we propose the utilization of synthetic faces as substitutes for real faces. Moreover, we introduce practical guidelines, including the establishment of thresholds and spot checking, to incorporate human-in-the-loop validation, thereby improving the accuracy and reliability of the deidentification process. In addition to this, this thesis also presents mitigation strategies to effectively handle reidentification risks. By considering the potential exploitation of soft biometric identifiers or non-biometric cues, we highlight the importance of implementing comprehensive measures such as robust data user licenses and privacy protection protocols.
Development of an Infrastructure Based Data Acquisition System to Naturalistically Collect the Roadway Environment
Sarkar, Abhijit; Papakis, Ioannis; Herbers, Eileen; Viray, Reginald (2023-12)
Automatic traffic monitoring is becoming an important investment for transportation specialists, especially as the overall volume of traffic continues to increase, as do crashes at intersections. Infrastructure cameras can be a good source of information for automatic monitoring of traffic situations at intersections. However, efficient computer vision methods that can process the video data effectively are required for this endeavor. Intersection cameras often record video data that are of low quality and low frame rate, making them challenging to use. In this project, we have demonstrated how traffic cameras can be used to automatically track roadway agents, find their kinematic behavior, and devise a safety measurement strategy, leveraging recent advancements in computer vision and deep learning. In this process, we have specifically focused on the Virginia Beach area and used publicly available traffic data to demonstrate our results. We developed a full computer vision pipeline that trains a custom object detector specifically using traffic data. We also used an optical flow method and a graph neural network to improve the accuracy of object tracking. The tracked objects from the image frames were further used as a point source and mapped to their GPS locations. Finally, the speed of each object was calculated to understand the traffic dynamics and determine possible crash predictors. This information can be used to quickly alert traffic control operators to a specific intersection that likely needs their attention so that crashes can be mitigated.
Enhancing Road Safety through Machine Learning for Prediction of Unsafe Driving Behaviors
Sonth, Akash Prakash (Virginia Tech, 2023-08-21)
Road accidents pose a significant threat, leading to fatalities and injuries with far-reaching consequences. This study addresses two crucial challenges in road safety: analyzing traffic intersections to enhance safety by predicting potentially risky situations, and monitoring driver activity to prevent distracted driving accidents. Focusing on Virginia's intersections, we thoroughly examine traffic participant interactions to identify and mitigate conflicts, employing graph-based modeling of traffic scenarios to evaluate contributing parameters. Additionally, we leverage graph neural networks to detect and track potential crash situations from intersection videos, offering practical recommendations to enhance intersection safety. To understand the causes of risky behavior, we specifically investigate accidents resulting from distracted driving, which has become more prevalent due to advanced driver assistance systems in semi-autonomous vehicles. For monitoring driver activity inside vehicles, we propose the use of Video Transformers on challenging secondary driver activity datasets, incorporating grayscale and low-quality data to overcome limitations in capturing overall image context. Finally, we validate our predictions by studying attention modules and introducing explainability into the computer vision model. This research contributes to improving road safety by providing comprehensive analysis and recommendations for intersection safety enhancement and prevention of distracted driving accidents.
Face De-identification of Drivers from NDS Data and Its Effectiveness in Human Factors
Thapa, Surendrabikram; Cook, Julie; Sarkar, Abhijit (National Surface Transportation Safety Center for Excellence, 2023-08-08)
Advancements in artificial intelligence (AI) and the Internet of Things (IoT) have made data the foundation for most technological innovations. As we embark on the era of big data analysis, broader access to quality data is essential for consistent advancement in research. Therefore, data sharing has become increasingly important in all fields, including transportation safety research. Data sharing can accelerate research by providing access to more data and the ability to replicate experiments and validate results. However, data sharing also poses challenges, such as the need to protect the privacy of research participants and address ethical and safety concerns when data contains personally identifiable information (PII). This report mainly focuses on the problem of sharing drivers’ face videos for transportation research. Driver video collected either through naturalistic driving studies (NDS) or simulator-based experiments contains useful information for driver behavior and human factors research. The report first gives an overview of the multitude of problems that are associated with sharing driver videos. Then, it demonstrates possible directions for data sharing by de-identifying drivers’ faces using AI-based techniques. We have demonstrated that recent developments in generative adversarial networks (GANs) can effectively help in de-identifying a person by swapping their face with that of another person. The results achieved through the proposed techniques were then evaluated qualitatively and quantitatively to prove the validity of such a system. Specifically, the report demonstrates how face-swapping algorithms can effectively de-identify faces while still preserving important attributes related to human factors research, including eye movements, head movements, and mouth movements. The experiments were done to assess the validity of GAN-based face de-identification on faces with varied anthropometric measurements. The participants used in the data had varied physical features as well. The dataset used was under lighting conditions that varied from normal to extreme conditions. This helped to check the robustness of the GAN-based techniques. The experiment was carried out for over 115,000 frames to account for most naturalistic driving conditions. Error metrics for head movements like differences in roll angle, pitch angle, and yaw angle were calculated. Similarly, the errors in eye aspect ratio, lip aspect ratio, and pupil circularity were also calculated as they are important in the assessment of various secondary behaviors of drivers while driving. We also calculated errors to assess the de-identified and original pairs more quantitatively. Next, we showed that a face can be swapped with faces that are artificially generated. We used GAN-based techniques to generate faces that were not present in the dataset used for training the model and were not known to exist before the generation process. These faces were then used for swapping with the original faces in our experiments. This gives researchers additional flexibility in choosing the type of face they want to swap. The report concludes by discussing possible measures to share such de-identified videos with the greater research community. Data sharing across disciplines helps to build collaboration and advance research, but it is important to ensure that ethical and safety concerns are addressed when data contains PII. The proposed techniques in this report provide a way to share driver face videos while still protecting the privacy of research participants; however, we recommend that such sharing should still be done under proper guidance from institutional review boards and should have a proper data use license.
A Graph Convolutional Neural Network Based Approach for Object Tracking Using Augmented Detections With Optical Flow
Papakis, Ioannis (Virginia Tech, 2021-05-18)
This thesis presents a novel method for online Multi-Object Tracking (MOT) using Graph Convolutional Neural Network (GCNN) based feature extraction and end-to-end feature matching for object association. The Graph based approach incorporates both appearance and geometry of objects at past frames as well as the current frame into the task of feature learning. This new paradigm enables the network to leverage the "contextual" information of the geometry of objects and allows us to model the interactions among the features of multiple objects. Another central innovation of the proposed framework is the use of the Sinkhorn algorithm for end-to-end learning of the associations among objects during model training. The network is trained to predict object associations by taking into account constraints specific to the MOT task. Additionally, in order to increase the sensitivity of the object detector, a new approach is presented that propagates previous frame detections into each new frame using optical flow. These are treated as added object proposals which are then classified as objects. A new traffic monitoring dataset is also provided, which includes naturalistic video footage from current infrastructure cameras in Virginia Beach City with a variety of vehicle density and environment conditions. Experimental evaluation demonstrates the efficacy of the proposed approaches on the provided dataset and the popular MOT Challenge Benchmark.
Harnessing the Power of Self-Training for Gaze Point Estimation in Dual Camera Transportation Datasets
Bhagat, Hirva Alpesh (Virginia Tech, 2023-06-14)
This thesis proposes a novel approach for efficiently estimating gaze points in dual camera transportation datasets. Traditional methods for gaze point estimation are dependent on large amounts of labeled data, which can be both expensive and time-consuming to collect. Additionally, alignment and calibration of the two camera views present significant challenges. To overcome these limitations, this thesis investigates the use of self-learning techniques such as semi-supervised learning and self-training, which can reduce the need for labeled data while maintaining high accuracy. The proposed method is evaluated on the DGAZE dataset and achieves a 57.2\% improvement in performance compared to the previous methods. This approach can prove to be a valuable tool for studying visual attention in transportation research, leading to more cost-effective and efficient research in this field.
Modeling Driver Behavior During Automated Vehicle Platooning Failures
McDonald, Anthony D.; Sarkar, Abhijit; Hickman, Jeffrey S.; Alambeigi, Hananeh; Vogelpohl, Tobias; Markkula, Gustav (SAFE-D: Safety Through Disruption National University Transportation Center, 2021-01)
Automated vehicles (AVs) promise to revolutionize driving safety. Driver models can aid in achieving this promise by providing a tool for designers to ensure safe interactions between human drivers and AVs. In this project, we performed a literature review to identify important factors for AV takeover safety and promising models to capture these factors. We also conducted a driving simulation experiment to address a research gap in silent automation failures. Finally, we developed a series of models to predict driver decision-making, braking,and steering responses using crash/near-crash data from the SHRP 2 naturalistic driving study and a driving simulation experiment. The analyses highlight the importance of visual parameters (in particular, visual looming) in driver responses and interactions with AVs. The modeling analysis suggested that models based on visual looming captured driver responses better than traditional baseline reaction time and closed-loop models. Further,the analysis of SHRP 2 data showed that gaze eccentricity of the last glance plays a critical role in driver decision-making. With further development, including the integration of important factors in takeover performance identified in the literature review and refinement of the role of gaze eccentricity, these models could be a valuable tool for AV software designers.
A Naturalistic Driving Study for Lane Change Detection and Personalization
Lakhkar, Radhika Anandrao (Virginia Tech, 2023-01-05)
Driver Assistance and Autonomous Driving features are becoming nearly ubiquitous in new vehicles. The intent of the Driver Assistant features is to assist the driver in making safer decisions. The intent of Autonomous Driving features is to execute vehicle maneuvers, without human intervention, in a safe manner. The overall goal of Driver Assistance and Autonomous Driving features is to reduce accidents, injuries, and deaths with a comforting driving experience. However, different drivers can react differently to advanced automated driving technology. It is therefore important to consider and improve the adaptability of these advances based on driver behavior. In this thesis, a human-centric approach is adopted in order to provide an enriching driving experience. The thesis investigates the natural behavior of drivers when changing lanes in terms of preferences of vehicle kinematics parameters using a real-world driving dataset collected as part of the Second Strategic Highway Research Program (SHRP2). The SHRP2 Naturalistic Driving Study (NDS) set is mined for lane change events. This work develops a way to detect reliable lane changing instances from a huge NDS dataset with more than 5,400,000 data files. The lane changing instances are distinguished from noisy and erroneous data by using machine vision lane tracking system variables such as left lane marker probability and right lane marker probability. We have shown that detected lane changing instances can be validated using only vehicle kinematics data. Kinematic vehicle parameters such as vehicle speed, lateral displacement, lateral acceleration, steering wheel angle, and lane change duration are then extracted and examined from time series data to characterize these lane-changing instances for a given driver. We have shown how these vehicle kinematic parameters change and exhibit patterns during lane change maneuvers for a specific driver. The thesis shows the limitations of analyzing vehicle kinematic parameters separately and develops a novel metric, Lane Change Dynamic Score(LCDS) that shows the collective effect of these vehicle kinematic parameters. LCDS is used to classify each lane change and thereby different driving styles.
Real-time Risk Prediction at Signalized Intersections Using a Graph Neural Network
Sonth, Akash; Sarkar, Abhijit; Jain, Sparsh; Bhagat, Hirva; Doerzaph, Zachary R. (Safe-D University Transportation Center, 2023-12)
Intersection-related traffic crashes and fatalities are major concerns for road safety. This project aimed to understand the major causes of conflicts at intersections by studying the intricate interplay between roadway agents. The approach involved using the current traffic camera systems to automatically process traffic video data. As manual annotation of video datasets is a very labor-intensive and costly process, this research leveraged modern computer vision algorithms to automatically process these videos and retrieve kinematic behavior of the traffic actors. Results demonstrated how traffic actors and road segments can be modeled independently via graphs and how they can be integrated into a framework that can model traffic systems. The team used a graph neural network to model (a) the interaction of all the roadway agents at any given instance and (b) their role in road safety, both individually and as a composite system. The model reports a near-real-time risk score for a traffic scene. The study concludes with a presentation of a new drone-based trajectory dataset to accelerate research in intersection safety.
Real-time Risk Prediction Using Temporal Gaze Pattern and Evidence Accumulation
Yang, Gary; Kaskar, Omkar; Sarkar, Abhijit (National Surface Transportation Safety Center for Excellence, 2024-06-26)
Driver gaze information provides insight into the driver’s perception and decision-making process in a dynamic driving environment. It is important to learn whether driver gaze information immediately before a safety-critical event (SCE) can be used to assess crash risk, and if so, how early in the crash we can predict the potential crash risk. This requires a thorough understanding of the temporal pattern of the data. In this project, we explored multiple key research questions pertaining to driver gaze and its relation to SCEs. We first looked at how temporal gaze patterns in SCEs like crashes and near-crashes are different from baseline events. Then we extended this analysis to understand if the temporal gaze pattern can indicate the direction of the threat. Finally, we showed how contextual information can aid the understanding of gaze patterns. We used traffic density, positions of traffic actors, and vehicle speed as key factors that can influence the temporal nature of gaze and impact safety. This study introduces a real-time crash risk prediction framework leveraging deep learning models based on driver gaze and contextual data. The analysis utilizes the Second Strategic Highway Research Program (SHRP 2) Naturalistic Driving Study, preprocessing it to extract temporal driver gaze patterns and contextual data for two distinct subsets of data. Dataset 1 comprises one-dimensional temporal driver gaze patterns only (named “SHRP2_Gaze”). Dataset 2 includes additional contextual information such as vehicle speed, distance to lead vehicle, and level of service (named “SHRP2_Gaze_Context).” Dataset 1 was used to explore the feasibility of predicting crash risk solely based on driver gaze data. Dataset 2 was applied to assessing the potential for early crash warnings by analyzing both driver gaze patterns and contextual data. For SHRP2_Gaze, four deep learning models (long short-term memory [LSTM], one-dimensional convolutional neural network [1D CNN], 1D CNN+LSTM, and 1D CNN+LSTM+Attention) were trained and tested on various inputs (Input 1: 20 seconds from five zones; Input 2: different lengths from five zones; and Input 3: 20 seconds from 19 zones), considering different lengths and zones of driver gaze data. The 1D CNN with XGBoost exhibited superior performance in crash risk prediction but demonstrated challenges in distinguishing specific crash and near-crash events. This portion of the study emphasized the significance of the temporal gaze pattern length and zone selection for real-time crash risk prediction using driver gaze data only. With SHRP2_Gaze_Context, the 1D CNN model with XGBoost emerged as the top-performing model when utilizing 11 types of 20-second driver gaze and contextual data to predict crash, near-crash, and baseline events. Notably, vehicle speed proves highly correlated with event categories, showcasing the effectiveness of combining driver gaze and contextual data. The study also highlighted the model’s ability to adapt to early crash warning scenarios, leveraging different 2-second multivariate data without an increase in computing time. The deep learning models employed in this study outperform traditional statistical models in distinguishing features from driver gaze and contextual data. These findings hold significant implications for accurate and efficient real-time crash risk prediction and early crash warning systems. The study’s insights cater to public agencies, insurance companies, and fleets, providing valuable understanding of driver behavior and contextual information leading to diverse safety events.
A Temporal Encoder-Decoder Approach to Extracting Blood Volume Pulse Signal Morphology from Face Videos
Li, Fulan (Virginia Tech, 2023-07-05)
This thesis considers methods for extracting blood volume pulse (BVP) representations from video of the human face. Whereas most previous systems have been concerned with estimating vital signs such as average heart rate, this thesis addresses the more difficult problem of recovering BVP signal morphology. We present a new approach that is inspired by temporal encoder-decoder architectures that have been used for audio signal separation. As input, this system accepts a temporal sequence of RGB (red, green, blue) values that have been spatially averaged over a small portion of the face. The output of the system is a temporal sequence that approximates a BVP signal. In order to reduce noise in the recovered signal, a separate processing step extracts individual pulses and performs normalization and outlier removal. After these steps, individual pulse shapes have been extracted that are sufficiently distinct to support biometric authentication. Our findings demonstrate the effectiveness of our approach in extracting BVP signal morphology from facial videos, which presents exciting opportunities for further research in this area. The source code is available at https://github.com/Adleof/CVPM-2023-Temporal-Encoder-Decoder-iPPG

Browsing by Author "Sarkar, Abhijit"

Results Per Page

Sort Options