Average precision (AP), the area under the recall-precision (RP) curve, isthe standard performance measure for object detection. Despite its wideacceptance, it has a number of shortcomings, the most important of which are(i) the inability to distinguish very different RP curves, and (ii) the lack ofdirectly measuring bounding box localization accuracy. In this paper, wepropose 'Localization Recall Precision (LRP) Error', a new metric which wespecifically designed for object detection. LRP Error is composed of threecomponents related to localization, false negative (FN) rate and false positive(FP) rate. Based on LRP, we introduce the 'Optimal LRP', the minimum achievableLRP error representing the best achievable configuration of the detector interms of recall-precision and the tightness of the boxes. In contrast to AP,which considers precisions over the entire recall domain, Optimal LRPdetermines the 'best' confidence score threshold for a class, which balancesthe trade-off between localization and recall-precision. In our experiments, weshow that, for state-of-the-art object (SOTA) detectors, Optimal LRP providesricher and more discriminative information than AP. We also demonstrate thatthe best confidence score thresholds vary significantly among classes anddetectors. Moreover, we present LRP results of a simple online video objectdetector which uses a SOTA still image object detector and show that theclass-specific optimized thresholds increase the accuracy against the commonapproach of using a general threshold for all classes. We provide the sourcecode that can compute LRP for the PASCAL VOC and MSCOCO datasets inhttps://github.com/cancam/LRP. Our source code can easily be adapted to otherdatasets as well.