TY - JOUR
T1 - Can we trust your explanations? Sanity checks for interpreters in android malware analysis
AU - Fan, Ming
AU - Wei, Wenying
AU - Xie, Xiaofei
AU - Liu, Yang
AU - Guan, Xiaohong
AU - Liu, Ting
N1 - Funding Information:
Manuscript received March 15, 2020; revised July 9, 2020; accepted August 12, 2020. Date of publication September 4, 2020; date of current version September 30, 2020. This work was supported in part by the National Key Research and Development Program of China under Grant 2016YFB1000903; in part by the National Natural Science Foundation of China under Grant 61902306, Grant 61532004, Grant 61532015, Grant 61632015, Grant 61602369, Grant U1766215, Grant 61772408, Grant 61702414, and Grant 61833015; in part by the China Postdoctoral Science Foundation under Grant 2019TQ0251 and Grant 2020M673439; in part by the Innovative Research Group of the National Natural Science Foundation of China under Grant 61721002; in part by the Ministry of Education Innovation Research Team (IRT_17R86); in part by the consulting research project of Chinese academy of engineering “The Online and Offline Mixed Educational Service System for ‘The Belt and Road’ Training in MOOC China” and Project of China Knowledge Centre for Engineering Science and Technology; in part by the Key Research Program of State Grid Shaanxi Electric Power Company; and in part by the National Research Foundation, Prime Minister’s Office, Singapore, under its National Cybersecurity Research and Development Program (Award No. NRF2018NCR-NCR005-0001), NRF Investiga-torship NRFI06-2020-0022. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Lorenzo Cavallaro. (Corresponding author: Ming Fan.) Ming Fan, Wenying Wei, Xiaohong Guan, and Ting Liu are with the School of Cyber Science and Engineering, MoEKLINNS, Xi’an Jiaotong University, Xi’an 710049, China (e-mail: mingfan@mail.xjtu.edu.cn).
Publisher Copyright:
© 2005-2012 IEEE.
PY - 2021
Y1 - 2021
N2 - With the rapid growth of Android malware, many machine learning-based malware analysis approaches are proposed to mitigate the severe phenomenon. However, such classifiers are opaque, non-intuitive, and difficult for analysts to understand the inner decision reason. For this reason, a variety of explanation approaches are proposed to interpret predictions by providing important features. Unfortunately, the explanation results obtained in the malware analysis domain cannot achieve a consensus in general, which makes the analysts confused about whether they can trust such results. In this work, we propose principled guidelines to assess the quality of five explanation approaches by designing three critical quantitative metrics to measure their stability, robustness, and effectiveness. Furthermore, we collect five widely-used malware datasets and apply the explanation approaches on them in two tasks, including malware detection and familial identification. Based on the generated explanation results, we conduct a sanity check of such explanation approaches in terms of the three metrics. The results demonstrate that our metrics can assess the explanation approaches and help us obtain the knowledge of most typical malicious behaviors for malware analysis.
AB - With the rapid growth of Android malware, many machine learning-based malware analysis approaches are proposed to mitigate the severe phenomenon. However, such classifiers are opaque, non-intuitive, and difficult for analysts to understand the inner decision reason. For this reason, a variety of explanation approaches are proposed to interpret predictions by providing important features. Unfortunately, the explanation results obtained in the malware analysis domain cannot achieve a consensus in general, which makes the analysts confused about whether they can trust such results. In this work, we propose principled guidelines to assess the quality of five explanation approaches by designing three critical quantitative metrics to measure their stability, robustness, and effectiveness. Furthermore, we collect five widely-used malware datasets and apply the explanation approaches on them in two tasks, including malware detection and familial identification. Based on the generated explanation results, we conduct a sanity check of such explanation approaches in terms of the three metrics. The results demonstrate that our metrics can assess the explanation approaches and help us obtain the knowledge of most typical malicious behaviors for malware analysis.
UR - http://www.scopus.com/inward/record.url?scp=85090937080&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85090937080&partnerID=8YFLogxK
U2 - 10.1109/TIFS.2020.3021924
DO - 10.1109/TIFS.2020.3021924
M3 - Article
AN - SCOPUS:85090937080
VL - 16
SP - 838
EP - 853
JO - IEEE Transactions on Information Forensics and Security
JF - IEEE Transactions on Information Forensics and Security
SN - 1556-6013
M1 - 9186721
ER -