Abstract
One-stage object detectors are trained by optimizing classification-loss andlocalization-loss simultaneously, with the former suffering much from extremeforeground-background class imbalance issue due to the large number of anchors.This paper alleviates this issue by proposing a novel framework to replace theclassification task in one-stage detectors with a ranking task, and adoptingthe Average-Precision loss (AP-loss) for the ranking problem. Due to itsnon-differentiability and non-convexity, the AP-loss cannot be optimizeddirectly. For this purpose, we develop a novel optimization algorithm, whichseamlessly combines the error-driven update scheme in perceptron learning andbackpropagation algorithm in deep networks. We verify good convergence propertyof the proposed algorithm theoretically and empirically. Experimental resultsdemonstrate notable performance improvement in state-of-the-art one-stagedetectors based on AP-loss over different kinds of classification-losses onvarious benchmarks, without changing the network architectures. Code isavailable at https://github.com/cccorn/AP-loss.