Abstract
Data imbalance, in which a plurality of the data samples come from a smallproportion of labels, poses a challenge in training deep neural networks.Unlike classification, in regression the labels are continuous, potentiallyboundless, and form a natural ordering. These distinct features of regressioncall for new techniques that leverage the additional information encoded inlabel-space relationships. This paper presents the RankSim (ranking similarity)regularizer for deep imbalanced regression, which encodes an inductive biasthat samples that are closer in label space should also be closer in featurespace. In contrast to recent distribution smoothing based approaches, RankSimcaptures both nearby and distant relationships: for a given data sample,RankSim encourages the sorted list of its neighbors in label space to match thesorted list of its neighbors in feature space. RankSim is complementary toconventional imbalanced learning techniques, including re-weighting, two-stagetraining, and distribution smoothing, and lifts the state-of-the-artperformance on three imbalanced regression benchmarks: IMDB-WIKI-DIR,AgeDB-DIR, and STS-B-DIR.