One pixel and three pixel attacks are pretty hard to get it seems
sayakpaul opened this issue · comments
Hey, @Hyperparticle.
Such a nice collection of materials, thank you!
I tried extending the CIFAR10 example to do some comparisons. It appears to me that for the kind of computational budget you followed, it's pretty hard to get a successful attack even on a small CNN (15722 learnable parameters).
Here's my notebook that does the comparison - https://colab.research.google.com/drive/1TKxtY63dqcuWAvrrDaDx3PQ3M7_xntQr?usp=sharing.
Am I missing out on something? One of the things I have changed is I have scaled the pixel values to be in the range of [0, 1]. Any help is much appreciated. Thanks!
There might be a lot of reasons why the attack might not work so well. For one, this implementation isn't particularly optimized for high performance, so for harder examples/networks it might take a long time to perform any successful attacks. The original authors' implementation seems to work better, I think because there are some additional details that are not mentioned in the paper (but it looks like they removed their implementation, unfortunately).
I see. But there does not seem to be a whole lot of differences between your implementation & the implementation I referenced in the Colab Notebook (most of which is referred from yours in fact).
But anyway, thank you for your inputs.