Abstract
Convolutional neural networks (CNNs) excel in computer vision but aresusceptible to adversarial attacks, crafted perturbations designed to misleadpredictions. Despite advances in adversarial training, a gap persists betweenmodel accuracy and robustness. To mitigate this issue, in this paper, wepresent a multi-teacher adversarial robustness distillation using an adaptivelearning strategy. Specifically, our proposed method first trained multipleclones of a baseline CNN model using an adversarial training strategy on a poolof perturbed data acquired through different adversarial attacks. Once trained,these adversarially trained models are used as teacher models to supervise thelearning of a student model on clean data using multi-teacher knowledgedistillation. To ensure an effective robustness distillation, we design anadaptive learning strategy that controls the knowledge contribution of eachmodel by assigning weights as per their prediction precision. Distillingknowledge from adversarially pre-trained teacher models not only enhances thelearning capabilities of the student model but also empowers it with thecapacity to withstand different adversarial attacks, despite having no exposureto adversarial data. To verify our claims, we extensively evaluated ourproposed method on MNIST-Digits and Fashion-MNIST datasets across diverseexperimental settings. The obtained results exhibit the efficacy of ourmulti-teacher adversarial distillation and adaptive learning strategy,enhancing CNNs' adversarial robustness against various adversarial attacks.