Abstract
Sample selection is the most straightforward technique to combat label noise,aiming to distinguish mislabeled samples during training and avoid thedegradation of the robustness of the model. In the workflow, $\textit{selectingpossibly clean data}$ and $\textit{model update}$ are iterative. However, theirinterplay and intrinsic characteristics hinder the robustness and efficiency oflearning with noisy labels: 1) The model chooses clean data with selectionbias, leading to the accumulated error in the model update. 2) Most selectionstrategies leverage partner networks or supplementary information to mitigatelabel corruption, albeit with increased computation resources and lowerthroughput speed. Therefore, we employ only one network with the jump mannerupdate to decouple the interplay and mine more semantic information from theloss for a more precise selection. Specifically, the selection of clean datafor each model update is based on one of the prior models, excluding the lastiteration. The strategy of model update exhibits a jump behavior in the form.Moreover, we map the outputs of the network and labels into the same semanticfeature space, respectively. In this space, a detailed and simple lossdistribution is generated to distinguish clean samples more effectively. Ourproposed approach achieves almost up to $2.53\times$ speedup, $0.46\times$ peakmemory footprint, and superior robustness over state-of-the-art works withvarious noise settings.