End to End Learning for Self-Driving Cars

by   Mariusz Bojarski, et al.

We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands. This end-to-end approach proved surprisingly powerful. With minimum training data from humans the system learns to drive in traffic on local roads with or without lane markings and on highways. It also operates in areas with unclear visual guidance such as in parking lots and on unpaved roads. The system automatically learns internal representations of the necessary processing steps such as detecting useful road features with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. Compared to explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously. We argue that this will eventually lead to better performance and smaller systems. Better performance will result because the internal components self-optimize to maximize overall system performance, instead of optimizing human-selected intermediate criteria, e.g., lane detection. Such criteria understandably are selected for ease of human interpretation which doesn't automatically guarantee maximum system performance. Smaller networks are possible because the system learns to solve the problem with the minimal number of processing steps. We used an NVIDIA DevBox and Torch 7 for training and an NVIDIA DRIVE(TM) PX self-driving car computer also running Torch 7 for determining where to drive. The system operates at 30 frames per second (FPS).


page 3

page 5

page 7

page 8


Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

As part of a complete software stack for autonomous driving, NVIDIA has ...

A Convolutional Neural Network Approach Towards Self-Driving Cars

A convolutional neural network (CNN) approach is used to implement a lev...

Ignition: An End-to-End Supervised Model for Training Simulated Self-Driving Vehicles

We introduce Ignition: an end-to-end neural network architecture for tra...

Driving Experience Transfer Method for End-to-End Control of Self-Driving Cars

In this paper, we present a transfer learning method for the end-to-end ...

Deep Reinforcement Learning with Mixed Convolutional Network

Recent research has shown that map raw pixels from a single front-facing...

Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction

Online lane graph construction is a promising but challenging task in au...

Robust Lane Detection through Self Pre-training with Masked Sequential Autoencoders and Fine-tuning with Customized PolyLoss

Lane detection is crucial for vehicle localization which makes it the fo...

Please sign up or login with your details

Forgot password? Click here to reset