Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

by   Mariusz Bojarski, et al.

As part of a complete software stack for autonomous driving, NVIDIA has created a neural-network-based system, known as PilotNet, which outputs steering angles given images of the road ahead. PilotNet is trained using road images paired with the steering angles generated by a human driving a data-collection car. It derives the necessary domain knowledge by observing human drivers. This eliminates the need for human engineers to anticipate what is important in an image and foresee all the necessary rules for safe driving. Road tests demonstrated that PilotNet can successfully perform lane keeping in a wide variety of driving conditions, regardless of whether lane markings are present or not. The goal of the work described here is to explain what PilotNet learns and how it makes its decisions. To this end we developed a method for determining which elements in the road image most influence PilotNet's steering decision. Results show that PilotNet indeed learns to recognize relevant objects on the road. In addition to learning the obvious features such as lane markings, edges of roads, and other cars, PilotNet learns more subtle features that would be hard to anticipate and program by engineers, for example, bushes lining the edge of the road and atypical vehicle classes.


page 2

page 3

page 5

page 6

page 7


End to End Video Segmentation for Driving : Lane Detection For Autonomous Car

Safety and decline of road traffic accidents remain important issues of ...

End to End Learning for Self-Driving Cars

We trained a convolutional neural network (CNN) to map raw pixels from a...

Multimodal End-to-End Learning for Autonomous Steering in Adverse Road and Weather Conditions

Autonomous driving is challenging in adverse road and weather conditions...

End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving

We propose an end-to-end three-task convolutional neural network (3TCNN)...

Semantic Label Reduction Techniques for Autonomous Driving

Semantic segmentation maps can be used as input to models for maneuverin...

Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars

As an initial assessment, over 480,000 labeled virtual images of normal ...

End-to-end Learning of Image based Lane-Change Decision

We propose an image based end-to-end learning framework that helps lane-...

Please sign up or login with your details

Forgot password? Click here to reset