Localization Recall Precision (LRP): A New Performance Metric for Object Detection

by   Kemal Oksuz, et al.

Average precision (AP), the area under the recall-precision (RP) curve, is the standard performance measure for object detection. Despite its wide acceptance, it has a number of shortcomings, the most important of which are (i) the inability to distinguish very different RP curves, and (ii) the lack of directly measuring bounding box localization accuracy. In this paper, we propose 'Localization Recall Precision (LRP) Error', a new metric which we specifically designed for object detection. LRP Error is composed of three components related to localization, false negative (FN) rate and false positive (FP) rate. Based on LRP, we introduce the 'Optimal LRP', the minimum achievable LRP error representing the best achievable configuration of the detector in terms of recall-precision and the tightness of the boxes. In contrast to AP, which considers precisions over the entire recall domain, Optimal LRP determines the 'best' confidence score threshold for a class, which balances the trade-off between localization and recall-precision. In our experiments, we show that, for state-of-the-art object (SOTA) detectors, Optimal LRP provides richer and more discriminative information than AP. We also demonstrate that the best confidence score thresholds vary significantly among classes and detectors. Moreover, we present LRP results of a simple online video object detector which uses a SOTA still image object detector and show that the class-specific optimized thresholds increase the accuracy against the common approach of using a general threshold for all classes. We provide the source code that can compute LRP for the PASCAL VOC and MSCOCO datasets in https://github.com/cancam/LRP. Our source code can easily be adapted to other datasets as well.


One Metric to Measure them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks

Despite being widely used as a performance measure for visual detection ...

Implicit Rate-Constrained Optimization of Non-decomposable Objectives

We consider a popular family of constrained optimization problems arisin...

A Study on Evaluation Standard for Automatic Crack Detection Regard the Random Fractal

A reasonable evaluation standard underlies construction of effective dee...

A Delay Metric for Video Object Detection: What Average Precision Fails to Tell

Average precision (AP) is a widely used metric to evaluate detection acc...

Seeing without Looking: Contextual Rescoring of Object Detections for AP Maximization

The majority of current object detectors lack context: class predictions...

Sensitivity of Average Precision to Bounding Box Perturbations

Object detection is a fundamental vision task. It has been highly resear...

Diagnosing Errors in Video Relation Detectors

Video relation detection forms a new and challenging problem in computer...

Please sign up or login with your details

Forgot password? Click here to reset