A Sensor Fusion and Localization System for Improving Vehicle Safety in Challenging Weather Conditions

SAE Level 5 autonomy requires the autonomous vehicle to be able to accurately sense the environment and detect obstacles in all weather and visibility conditions. This sensing problem becomes significantly challenging in weather conditions that include such events as sudden change in lighting, smoke, fog, snow, and rain. There is no standalone sensor currently on the market that can provide reliable perception data in all conditions. We demonstrate that a combination of Long Wave Infrared (LWIR) cameras with radar provide a viable sensing system that is robust to adverse visibility conditions. We have validated this prototype system both in simulation as well as in real-world traffic using a 2017 Lincoln MKZ operating in College Station, TX.