DeepGauge: Multi-granularity testing criteria for deep learning systems

Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang

研究成果: Chapter in Book/Report/Conference proceedingConference contribution

179 被引用数 (Scopus)

抄録

Deep learning (DL) defines a new data-driven programming paradigm that constructs the internal system logic of a crafted neuron network through a set of training data. We have seen wide adoption of DL in many safety-critical scenarios. However, a plethora of studies have shown that the state-of-the-art DL systems suffer from various vulnerabilities which can lead to severe consequences when applied to real-world applications. Currently, the testing adequacy of a DL system is usually measured by the accuracy of test data. Considering the limitation of accessible high quality test data, good accuracy performance on test data can hardly provide confidence to the testing adequacy and generality of DL systems. Unlike traditional software systems that have clear and controllable logic and functionality, the lack of interpretability in a DL system makes system analysis and defect detection difficult, which could potentially hinder its real-world deployment. In this paper, we propose DeepGauge, a set of multi-granularity testing criteria for DL systems, which aims at rendering a multi-faceted portrayal of the testbed. The in-depth evaluation of our proposed testing criteria is demonstrated on two well-known datasets, five DL systems, and with four state-of-the-art adversarial attack techniques against DL. The potential usefulness of DeepGauge sheds light on the construction of more generic and robust DL systems.

本文言語英語
ホスト出版物のタイトルASE 2018 - Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering
編集者Christian Kastner, Marianne Huchard, Gordon Fraser
出版社Association for Computing Machinery, Inc
ページ120-131
ページ数12
ISBN(電子版)9781450359375
DOI
出版ステータス出版済み - 9 3 2018
イベント33rd IEEE/ACM International Conference on Automated Software Engineering, ASE 2018 - Montpellier, フランス
継続期間: 9 3 20189 7 2018

出版物シリーズ

名前ASE 2018 - Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

その他

その他33rd IEEE/ACM International Conference on Automated Software Engineering, ASE 2018
国/地域フランス
CityMontpellier
Period9/3/189/7/18

All Science Journal Classification (ASJC) codes

  • 計算理論と計算数学
  • 人間とコンピュータの相互作用
  • ソフトウェア

フィンガープリント

「DeepGauge: Multi-granularity testing criteria for deep learning systems」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル