Abstract
Vision Transformers (ViTs) have a radically different architecture withsignificantly less inductive bias than Convolutional Neural Networks. Alongwith the improvement in performance, security and robustness of ViTs are alsoof great importance to study. In contrast to many recent works that exploit therobustness of ViTs against adversarial examples, this paper investigates arepresentative causative attack, i.e., backdoor. We first examine thevulnerability of ViTs against various backdoor attacks and find that ViTs arealso quite vulnerable to existing attacks. However, we observe that theclean-data accuracy and backdoor attack success rate of ViTs responddistinctively to patch transformations before the positional encoding. Then,based on this finding, we propose an effective method for ViTs to defend bothpatch-based and blending-based trigger backdoor attacks via patch processing.The performances are evaluated on several benchmark datasets, includingCIFAR10, GTSRB, and TinyImageNet, which show the proposed novel defense is verysuccessful in mitigating backdoor attacks for ViTs. To the best of ourknowledge, this paper presents the first defensive strategy that utilizes aunique characteristic of ViTs against backdoor attacks.