Stealing Deep Reinforcement Learning Models for Fun and Profit

Kangjie Chen, Shangwei Guo, Tianwei Zhang, Xiaofei Xie, Yang Liu

研究成果: Chapter in Book/Report/Conference proceedingConference contribution

抄録

This paper presents the first model extraction attack against Deep Reinforcement Learning (DRL), which enables an external adversary to precisely recover a black-box DRL model only from its interaction with the environment. Model extraction attacks against supervised Deep Learning models have been widely studied. However, those techniques cannot be applied to the reinforcement learning scenario due to DRL models' high complexity, stochasticity and limited observable information. We propose a novel methodology to overcome the above challenges. The key insight of our approach is that the process of DRL model extraction is equivalent to imitation learning, a well-established solution to learn sequential decision-making policies. Based on this observation, our methodology first builds a classifier to reveal the training algorithm family of the targeted black-box DRL model only based on its predicted actions, and then leverages state-of-the-art imitation learning techniques to replicate the model from the identified algorithm family. Experimental results indicate that our methodology can effectively recover the DRL models with high fidelity and accuracy. We also demonstrate two use cases to show that our model extraction attack can (1) significantly improve the success rate of adversarial attacks, and (2) steal DRL models stealthily even they are protected by DNN watermarks. These pose a severe threat to the intellectual property and privacy protection of DRL applications.

本文言語英語
ホスト出版物のタイトルASIA CCS 2021 - Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security
出版社Association for Computing Machinery, Inc
ページ307-319
ページ数13
ISBN(電子版)9781450382878
DOI
出版ステータス出版済み - 5 24 2021
外部発表はい
イベント16th ACM Asia Conference on Computer and Communications Security, ASIA CCS 2021 - Virtual, Online, 香港
継続期間: 6 7 20216 11 2021

出版物シリーズ

名前ASIA CCS 2021 - Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security

会議

会議16th ACM Asia Conference on Computer and Communications Security, ASIA CCS 2021
国/地域香港
CityVirtual, Online
Period6/7/216/11/21

All Science Journal Classification (ASJC) codes

  • コンピュータ ネットワークおよび通信
  • コンピュータ サイエンスの応用
  • 情報システム
  • ソフトウェア

フィンガープリント

「Stealing Deep Reinforcement Learning Models for Fun and Profit」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル