ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling

by   Quanyi Li, et al.

Large-scale driving datasets such as Waymo Open Dataset and nuScenes substantially accelerate autonomous driving research, especially for perception tasks such as 3D detection and trajectory forecasting. Since the driving logs in these datasets contain HD maps and detailed object annotations which accurately reflect the real-world complexity of traffic behaviors, we can harvest a massive number of complex traffic scenarios and recreate their digital twins in simulation. Compared to the hand-crafted scenarios often used in existing simulators, data-driven scenarios collected from the real world can facilitate many research opportunities in machine learning and autonomous driving. In this work, we present ScenarioNet, an open-source platform for large-scale traffic scenario modeling and simulation. ScenarioNet defines a unified scenario description format and collects a large-scale repository of real-world traffic scenarios from the heterogeneous data in various driving datasets including Waymo, nuScenes, Lyft L5, and nuPlan datasets. These scenarios can be further replayed and interacted with in multiple views from Bird-Eye-View layout to realistic 3D rendering in MetaDrive simulator. This provides a benchmark for evaluating the safety of autonomous driving stacks in simulation before their real-world deployment. We further demonstrate the strengths of ScenarioNet on large-scale scenario generation, imitation learning, and reinforcement learning in both single-agent and multi-agent settings. Code, demo videos, and website are available at


page 2

page 5

page 7

page 22

page 23

page 24

page 25

page 26


InterSim: Interactive Traffic Simulation via Explicit Relation Modeling

Interactive traffic simulation is crucial to autonomous driving systems ...

Automatic Traffic Scenario Conversion from OpenSCENARIO to CommonRoad

Scenarios are a crucial element for developing, testing, and verifying a...

BITS: Bi-level Imitation for Traffic Simulation

Simulation is the key to scaling up validation and verification for robo...

PeSOTIF: a Challenging Visual Dataset for Perception SOTIF Problems in Long-tail Traffic Scenarios

Perception algorithms in autonomous driving systems confront great chall...

TSTTC: A Large-Scale Dataset for Time-to-Contact Estimation in Driving Scenarios

Time-to-Contact (TTC) estimation is a critical task for assessing collis...

The OpenCDA Open-source Ecosystem for Cooperative Driving Automation Research

Advances in Single-vehicle intelligence of automated driving have encoun...

Learning to Drive Anywhere

Human drivers can seamlessly adapt their driving decisions across geogra...

Please sign up or login with your details

Forgot password? Click here to reset