One Metric to Measure them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks
share
Despite being widely used as a performance measure for visual detection tasks, Average Precision (AP) is limited in (i) including localisation quality, (ii) interpretability and (iii) applicability to outputs without confidence scores. Panoptic Quality (PQ), a measure proposed for evaluating panoptic segmentation (Kirillov et al., 2019), does not suffer from these limitations but is limited to panoptic segmentation. In this paper, we propose Localisation Recall Precision (LRP) Error as the performance measure for all visual detection tasks. LRP Error, initially proposed only for object detection by Oksuz et al. (2018), does not suffer from the aforementioned limitations and is applicable to all visual detection tasks. We also introduce Optimal LRP (oLRP) Error as the minimum LRP error obtained over confidence scores to evaluate visual detectors and obtain optimal thresholds for deployment. We provide a detailed comparative analysis of LRP with AP and PQ, and use 35 state-of-the-art visual detectors from four common visual detection tasks (i.e. object detection, keypoint detection, instance segmentation and panoptic segmentation) to empirically show that LRP provides richer and more discriminative information than its counterparts. Code available at: https://github.com/kemaloksuz/LRP-Error
READ FULL TEXT