PapeRman #5


On Evaluating Adversarial Robustness

Authors: Nicholas Carlini1, Anish Athalye2, Nicolas Papernot1, Wieland Brendel3, Jonas Rauber3,
Dimitris Tsipras2, Ian Goodfellow1, Aleksander Mądry2
Institutes: 1 Google Brain 2 MIT 3 University of Tübingen

Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that withstand adaptive attacks, few have succeeded; most papers that propose defenses are quickly shown to be incorrect.

We believe a large contributing factor is the difficulty of performing security evaluations. In this paper, we discuss the methodological foundations, review commonly accepted best practices, and suggest new methods for evaluating defenses to adversarial examples. We hope that both researchers developing defenses as well as readers and reviewers who wish to understand the completeness of an evaluation consider our advice in order to avoid common pitfalls.

This image has an empty alt attribute; its file name is screenshot-from-2019-01-17-01-49-57.png

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s