Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Cars

One of the major open challenges in self-driving cars is the ability to detect cars and pedestrians to safely navigate in the world. Deep learning-based object detector approaches have enabled great advances in using camera imagery to detect and classify objects. But for a safety critical application such as autonomous driving, the error rates of the current state-of-the-art are still too high to enable safe operation. Moreover, our characterization of object detector performance is primarily limited to testing on prerecorded datasets. Errors that occur on novel data go undetected without additional human labels. In this paper, we propose an automated method to identify mistakes made by object detectors without ground truth labels. We show that inconsistencies in object detector output between a pair of similar images can be used to identify false negatives(e.g. missed detections). In particular, we study two distinct cues - temporal and stereo inconsistencies - using data that is readily available on most autonomous vehicles. Our method can be used with any camera-based object detector and we evaluate the technique on several sets of real world data. The proposed method achieves over 97 automatically identifying missed detections produced by one of the leading state-of-the-art object detectors in the literature. We also release a new tracking dataset with over 100 sequences totaling more than 80,000 labeled images from a game engine to facilitate further research.


page 3

page 4

page 5

page 6

page 7


CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving

Contemporary deep-learning object detection methods for autonomous drivi...

Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection

Self-driving cars must detect other vehicles and pedestrians in 3D to pl...

Towards an Error-free Deep Occupancy Detector for Smart Camera Parking System

Although the smart camera parking system concept has existed for decades...

Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars

As an initial assessment, over 480,000 labeled virtual images of normal ...

EyeDAS: Securing Perception of Autonomous Cars Against the Stereoblindness Syndrome

The ability to detect whether an object is a 2D or 3D object is extremel...

SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectors

We present SHIFT3D, a differentiable pipeline for generating 3D shapes t...

Self-Learning Camera: Autonomous Adaptation of Object Detectors to Unlabeled Video Streams

Learning object detectors requires massive amounts of labeled training s...

Please sign up or login with your details

Forgot password? Click here to reset