One Pixel Attack for Fooling Deep Neural Networks

Research output: Contribution to journalArticle

41 Citations (Scopus)

Abstract

Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

Original languageEnglish
Article number8601309
Pages (from-to)828-841
Number of pages14
JournalIEEE Transactions on Evolutionary Computation
Volume23
Issue number5
DOIs
Publication statusPublished - Oct 2019

Fingerprint

Pixel
Pixels
Attack
Neural Networks
Differential Evolution
Learning systems
Machine Learning
Evolutionary algorithms
Scenarios
Neural networks
Evolutionary Computation
Black Box
Vulnerability
Small Perturbations
Confidence
Extremes
Deep neural networks
Costs
Robustness
Perturbation

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Computational Theory and Mathematics

Cite this

One Pixel Attack for Fooling Deep Neural Networks. / Su, Jiawei; Vargas, Danilo Vasconcellos; Sakurai, Kouichi.

In: IEEE Transactions on Evolutionary Computation, Vol. 23, No. 5, 8601309, 10.2019, p. 828-841.

Research output: Contribution to journalArticle

@article{13a144e39e39466ba04f6276e37033da,
title = "One Pixel Attack for Fooling Deep Neural Networks",
abstract = "Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97{\%} of the natural images in Kaggle CIFAR-10 test dataset and 16.04{\%} of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03{\%} and 22.91{\%} confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.",
author = "Jiawei Su and Vargas, {Danilo Vasconcellos} and Kouichi Sakurai",
year = "2019",
month = "10",
doi = "10.1109/TEVC.2019.2890858",
language = "English",
volume = "23",
pages = "828--841",
journal = "IEEE Transactions on Evolutionary Computation",
issn = "1089-778X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "5",

}

TY - JOUR

T1 - One Pixel Attack for Fooling Deep Neural Networks

AU - Su, Jiawei

AU - Vargas, Danilo Vasconcellos

AU - Sakurai, Kouichi

PY - 2019/10

Y1 - 2019/10

N2 - Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

AB - Recent research has revealed that the output of deep neural networks (DNNs) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified. For that we propose a novel method for generating one-pixel adversarial perturbations based on differential evolution (DE). It requires less adversarial information (a black-box attack) and can fool more types of networks due to the inherent features of DE. The results show that 67.97% of the natural images in Kaggle CIFAR-10 test dataset and 16.04% of the ImageNet (ILSVRC 2012) test images can be perturbed to at least one target class by modifying just one pixel with 74.03% and 22.91% confidence on average. We also show the same vulnerability on the original CIFAR-10 dataset. Thus, the proposed attack explores a different take on adversarial machine learning in an extreme limited scenario, showing that current DNNs are also vulnerable to such low dimension attacks. Besides, we also illustrate an important application of DE (or broadly speaking, evolutionary computation) in the domain of adversarial machine learning: creating tools that can effectively generate low-cost adversarial attacks against neural networks for evaluating robustness.

UR - http://www.scopus.com/inward/record.url?scp=85073072064&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073072064&partnerID=8YFLogxK

U2 - 10.1109/TEVC.2019.2890858

DO - 10.1109/TEVC.2019.2890858

M3 - Article

AN - SCOPUS:85073072064

VL - 23

SP - 828

EP - 841

JO - IEEE Transactions on Evolutionary Computation

JF - IEEE Transactions on Evolutionary Computation

SN - 1089-778X

IS - 5

M1 - 8601309

ER -