Abstract
Deep neural network models are massively deployed on a wide variety ofhardware platforms. This results in the appearance of new attack vectors thatsignificantly extend the standard attack surface, extensively studied by theadversarial machine learning community. One of the first attack that aims atdrastically dropping the performance of a model, by targeting its parameters(weights) stored in memory, is the Bit-Flip Attack (BFA). In this work, wepoint out several evaluation challenges related to the BFA. First of all, thelack of an adversary's budget in the standard threat model is problematic,especially when dealing with physical attacks. Moreover, since the BFA presentscritical variability, we discuss the influence of some training parameters andthe importance of the model architecture. This work is the first to present theimpact of the BFA against fully-connected architectures that present differentbehaviors compared to convolutional neural networks. These results highlightthe importance of defining robust and sound evaluation methodologies toproperly evaluate the dangers of parameter-based attacks as well as measure thereal level of robustness offered by a defense.