Camera-based Feature Identification for EasyMile Operation

The EasyMile deployment studied in this work included cameras that captured the 360 degrees of roadway environment around the vehicle. We developed a scene perception algorithm using computer vision technology to track other roadway agents like cars, pedestrians, and bicyclists around the EasyMile LSAV. We used object detection and tracking algorithms to track the trajectories of each of the roadway agents. Then we used perspective geometry and camera specifications to find the relative distances and speeds of these agents with respect to the EasyMile. This helped us understand the configurations of the traffic around the LSAV and study other drivers’ temporal behavior. For example, the collected data shows the approach of any vehicle towards the EasyMile. Finally, we used this information to study other vehicles’ maneuvers and show how the information from the cameras can be used to study simple maneuvers of other vehicles such as cut-ins, lane changes, and following behavior.

Through these camera-based tools, we have demonstrated examples from the real-world deployment. We studied following behavior characteristics that show the relative distance and speed of other vehicles’ following behavior. We have also demonstrated cut-in behaviors through the longitudinal and lateral trajectories of cut-in vehicles. We also showed how abrupt cut-ins may lead the EasyMile to apply its brakes, leading to safety critical events for following vehicles. Finally, we demonstrated how pedestrian behavior can be studied via these camera-based methods.