Harry24k / adversarial-attacks-pytorch

PyTorch implementation of adversarial attacks [torchattacks]

Home Page:https://adversarial-attacks-pytorch.readthedocs.io/en/latest/index.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[bug] Side effect of Attack.save() with int return type

Framartin opened this issue · comments

I am a bit confused about the management of return types of the base class torchattacks.attack.Attack.

  1. If the return type is set to int (_return_type=='int'), calling Attack.save() resets the return type to float (see line below). This may be a hard-to-detect side effect.

    self._return_type = 'float'

  2. The metrics in Attack.save() are computed on float adversarial examples and not on integer adversarial examples. It might lead to unexpected conclusions when comparing the printed accuracy and the accuracy of saved adversarial examples.

My proposition

I can create a PR with the following modifications:

  • Attack.save() does not modify _return_type of the class
  • Attack.save() computes metrics (L2 norm and accuracy) on scaled adversarial examples (adv_images/255) when _return_type=='int'.

@Harry24k Does it look good to you? Thanks for your time :)

Thank you for your comment :)
I fully understood what can be the problem, and I removed the modification during saving.
It will be updated in this commit.

Great! It looks good. Thanks for your time :)