Abstract
Learning adversarial examples can be formulated as an optimization problem ofmaximizing the loss function with some box-constraints. However, for solvingthis induced optimization problem, the state-of-the-art gradient-based methodssuch as FGSM, I-FGSM and MI-FGSM look different from their original methodsespecially in updating the direction, which makes it difficult to understandthem and then leaves some theoretical issues to be addressed in viewpoint ofoptimization. In this paper, from the perspective of adapting step-size, weprovide a unified theoretical interpretation of these gradient-basedadversarial learning methods. We show that each of these algorithms is in facta specific reformulation of their original gradient methods but using thestep-size rules with only current gradient information. Motivated by suchanalysis, we present a broad class of adaptive gradient-based algorithms basedon the regular gradient methods, in which the step-size strategy utilizinginformation of the accumulated gradients is integrated. Such adaptive step-sizestrategies directly normalize the scale of the gradients rather than use someempirical operations. The important benefit is that convergence for theiterative algorithms is guaranteed and then the whole optimization process canbe stabilized. The experiments demonstrate that our AdaI-FGM consistentlyoutperforms I-FGSM and AdaMI-FGM remains competitive with MI-FGSM for black-boxattacks.