Abstract
Recurrent Neural Networks (RNNs) have achieved tremendous success insequential data processing. However, it is quite challenging to interpret andverify RNNs' behaviors directly. To this end, many efforts have been made toextract finite automata from RNNs. Existing approaches such as exact learningare effective in extracting finite-state models to characterize the statedynamics of RNNs for formal languages, but are limited in the scalability toprocess natural languages. Compositional approaches that are scablable tonatural languages fall short in extraction precision. In this paper, weidentify the transition sparsity problem that heavily impacts the extractionprecision. To address this problem, we propose a transition rule extractionapproach, which is scalable to natural language processing models and effectivein improving extraction precision. Specifically, we propose an empirical methodto complement the missing rules in the transition diagram. In addition, wefurther adjust the transition matrices to enhance the context-aware ability ofthe extracted weighted finite automaton (WFA). Finally, we propose two dataaugmentation tactics to track more dynamic behaviors of the target RNN.Experiments on two popular natural language datasets show that our method canextract WFA from RNN for natural language processing with better precision thanexisting approaches. Our code is available athttps://github.com/weizeming/Extract_WFA_from_RNN_for_NL.