The article revisits the transferable adversarial examples, which are a significant security concern in real-world, black-box attack scenarios. The authors identify two main problems in common evaluation practices: lack of systematic, one-to-one attack comparison and fair hyperparameter settings, and lack of comparisons for attack stealthiness. To address these issues, they propose a new attack categorization strategy and conduct systematic and fair intra-category analyses on transferability. They also consider diverse imperceptibility metrics and finer-grained stealthiness characteristics from the perspective of attack traceback. The evaluation leads to several new insights, such as the performance of early attack methods and the false sense of security given by some state-of-the-art defenses.
Publication date: 19 Oct 2023
Project Page: https://github.com/ZhengyuZhao/TransferAttackEval
Paper: https://arxiv.org/pdf/2310.11850
