DeepMutation++: A mutation testing framework for deep learning systems

Qiang Hu, Lei Ma, Xiaofei Xie, Bing Yu, Yang Liu, Jianjun Zhao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep neural networks (DNNs) are increasingly expanding their real-world applications across domains, e.g., image processing, speech recognition and natural language processing. However, there is still limited tool support for DNN testing in terms of test data quality and model robustness. In this paper, we introduce a mutation testing-based tool for DNNs, DeepMutation++, which facilitates the DNN quality evaluation, supporting both feed-forward neural networks (FNNs) and stateful recurrent neural networks (RNNs). It not only enables to statically analyze the robustness of a DNN model against the input as a whole, but also allows to identify the vulnerable segments of a sequential input (e.g. audio input) by runtime analysis. It is worth noting that DeepMutation++ specially features the support of RNNs mutation testing. The tool demo video can be found on the project website https://sites.google.com/view/deepmutationpp.

Original languageEnglish
Title of host publicationProceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1158-1161
Number of pages4
ISBN (Electronic)9781728125084
DOIs
Publication statusPublished - Nov 2019
Event34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019 - San Diego, United States
Duration: Nov 10 2019Nov 15 2019

Publication series

NameProceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019

Conference

Conference34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019
CountryUnited States
CitySan Diego
Period11/10/1911/15/19

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software
  • Control and Optimization

Cite this

Hu, Q., Ma, L., Xie, X., Yu, B., Liu, Y., & Zhao, J. (2019). DeepMutation++: A mutation testing framework for deep learning systems. In Proceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019 (pp. 1158-1161). [8952248] (Proceedings - 2019 34th IEEE/ACM International Conference on Automated Software Engineering, ASE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASE.2019.00126