Assessing Effects of Object Detection Performance on Simulated Crash Outcomes for an Automated Driving System

Highly Automated Vehicles (AVs) have the capability to revolutionize the transportation system. These systems have the possibility to make roads safer as AVs do not have limitations that human drivers do, many of which are common causes of vehicle crashes (e.g., distraction or fatigue) often defined generically as human error. The deployment of AVs is likely to be very gradual however, and there will exist situations in which the AV will be driving in close proximity with human drivers across the foreseeable future. Given the persistent crash problem in which the makority of crashes are attributed to driver error, humans will continue to create potential collision scenarios that an AV will be expected to try and avoid or mitigate if developed appropriately. The absence of unreasonable risk in an AVs ability to comprehend and react in these situations is referred to as operational safety. Unlike advanced driver assistance systems (ADAS), highly automated vehicles are required to perform the entirety of the dynamic driving task (DDT) and have a greater responsibility to achieve a high level of operational safety. To address this concern, scenario-based testing has increasingly become a popular option for evaluating AV performance. On a functional level, an AV typically consists of three basic systems: the perception system, the decision and path planning system, and vehicle motion control system. A minimum level of performance is needed in each of these functional blocks to achieve an adequate level of operational safety. The goal of this study was to investigate the effects that perception system performance (i.e., target object state errors) has on vehicle operational safety in collision scenarios similar to that created by human drivers. In the first part of this study, recent annual crash data was used to define a relevant crash population of possible scenarios involving intersections that an AV operating as an urban taxi may encounter. Common crash maneuvers and characteristics were combined to create a set of testing scenarios that represent a high iii percentage of the overall crash population. In the second part of this study, each test scenario was executed using an AV test platform during closed road testing to determine possible real-world perception system performance. This provided a measure of the error in object detection measurements compared to the ideal (i.e., where a vehicle was detected to be compared to where it actually was). In the third part of this study, a set of vehicle simulations were performed to assess the effect of perception system performance on crash outcomes. This analysis simulated hypothetical crashes between an AV and one other collision partner. First an initial worst-case impact configuration was defined and was based on injury outcomes seen in crash data. The AV was then simulated to perform a variety of evasive maneuvers based on an adaptation of a non-impaired driver model. The impact location and orientation of the collision partner was simulated as two states: one based on the object detection of an ideal perception system and the other based on the object detection of the perception system from the AV platform used during the road testing. For simulations in which the two vehicles contacted each other, a planar momentum-impulse model was used for impact modeling and injury outcomes were predicted using an omni-directional injury model taken from recent literature. Results from this study indicate that errors in perception system measurements can change the perceived occupant injury risk within a crash. Sensitivity was found to be dependent on the specific crash type as well as what evasive maneuver is taken. Sensitivities occurred mainly due to changes in the principal direction of force for the crash and the interaction within the injury risk prediction curves. In order to achieve full operational safety, it will likely be important to understand the influence that each functional system (perception, decision, and control) may have on AV performance in these crash scenarios.