A Counterfactual Safety Margin Perspective on the Scoring of Autonomous Vehicles' Riskiness

by   Alessandro Zanardi, et al.

Autonomous Vehicles (AVs) have the potential to provide numerous societal benefits, such as decreased road accidents and increased overall transportation efficiency. However, quantifying the risk associated with AVs is challenging due to the lack of historical data and the rapidly evolving technology. This paper presents a data-driven framework for comparing the risk of different AVs' behaviors in various operational design domains (ODDs), based on counterfactual simulations of "misbehaving" road users. We introduce the concept of counterfactual safety margin, which represents the minimum deviation from normal behavior that could lead to a collision. This concept helps to find the most critical scenarios but also to assess the frequency and severity of risk of AVs. We show that the proposed methodology is applicable even when the AV's behavioral policy is unknown – through worst- and best-case analyses – making the method useful also to external third-party risk assessors. Our experimental results demonstrate the correlation between the safety margin, the driving policy quality, and the ODD shedding light on the relative risk associated with different AV providers. This work contributes to AV safety assessment and aids in addressing legislative and insurance concerns surrounding this emerging technology.


Risk-Based Safety Envelopes for Autonomous Vehicles Under Perception Uncertainty

Ensuring the safety of autonomous vehicles, given the uncertainty in sen...

Risk Measurement, Risk Entropy, and Autonomous Driving Risk Modeling

It has been for a long time to use big data of autonomous vehicles for p...

Methodology for Testing and Evaluation of Safety Analytics Approaches

There has been a significant increase in the development of data-driven ...

DeepGuard: A Framework for Safeguarding Autonomous Driving Systems from Inconsistent Behavior

The deep neural networks (DNNs)based autonomous driving systems (ADSs) a...

Microscopic Traffic Models, Accidents, and Insurance Losses

The paper develops a methodology to enable microscopic models of transpo...

Please sign up or login with your details

Forgot password? Click here to reset