Recurrent Neural Networks (RNNs) have achieved tremendous success in sequential data processing. However, it is quite challenging to interpret and verify RNNs' behaviors directly. To this end, many efforts have been made to extract finite automata from RNNs. Existing approaches such as exact learning are effective in extracting finite-state models to characterize the state dynamics of RNNs for formal languages, but are limited in the scalability to process natural languages. Compositional approaches that are scablable to natural languages fall short in extraction precision. In this paper, we identify the transition sparsity problem that heavily impacts the extraction precision. To address this problem, we propose a transition rule extraction approach, which is scalable to natural language processing models and effective in improving extraction precision. Specifically, we propose an empirical method to complement the missing rules in the transition diagram. In addition, we further adjust the transition matrices to enhance the context-aware ability of the extracted weighted finite automaton (WFA). Finally, we propose two data augmentation tactics to track more dynamic behaviors of the target RNN. Experiments on two popular natural language datasets show that our method can extract WFA from RNN for natural language processing with better precision than existing approaches.

本文提出了解决 RNN 的状态转移抽取中过稀疏问题的新方法，包括用经验方法补充缺失规则、调整转移矩阵以加强上下文感知、以及数据增强策略以跟踪目标 RNN 的更多动态行为。研究结果表明，我们的方法能够以更好的精度从 RNN 中提取加权有限自动机，特别适用于自然语言处理模型。

从循环神经网络中提取加权有限状态自动机以处理自然语言