Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks

Xiyue Zhang, Xiaoning Du, Xiaofei Xie, Lei Ma, Yang Liu, Meng Sun

研究成果: 書籍/レポート タイプへの寄稿会議への寄与

5 被引用数 (Scopus)

抄録

Recurrent Neural Networks (RNNs) have demonstrated their effectiveness in learning and processing sequential data (e.g., speech and natural language). However, due to the black-box nature of neural networks, understanding the decision logic of RNNs is quite challenging. Some recent progress has been made to approximate the behavior of an RNN by weighted automata. They provide better interpretability, but still suffer from poor scalability. In this paper, we propose a novel approach to extracting weighted automata with the guidance of a target RNN’s decision and context information. In particular, we identify the patterns of RNN’s step-wise predictive decisions to instruct the formation of automata states. Further, we propose a state composition method to enhance the context-awareness of the extracted model. Our in-depth evaluations on typical RNN tasks, including language model and classification, demonstrate the effectiveness and advantage of our method over the state-of-the-arts. The evaluation results show that our method can achieve accurate approximation of an RNN even on large-scale tasks.

本文言語英語
ホスト出版物のタイトル35th AAAI Conference on Artificial Intelligence, AAAI 2021
出版社Association for the Advancement of Artificial Intelligence
ページ11699-11707
ページ数9
ISBN(電子版)9781713835974
出版ステータス出版済み - 2021
イベント35th AAAI Conference on Artificial Intelligence, AAAI 2021 - Virtual, Online
継続期間: 2月 2 20212月 9 2021

出版物シリーズ

名前35th AAAI Conference on Artificial Intelligence, AAAI 2021
13A

会議

会議35th AAAI Conference on Artificial Intelligence, AAAI 2021
CityVirtual, Online
Period2/2/212/9/21

!!!All Science Journal Classification (ASJC) codes

  • 人工知能

フィンガープリント

「Decision-Guided Weighted Automata Extraction from Recurrent Neural Networks」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル