In order to reveal the rationale behind model predictions, many works have
exploited providing explanations in various forms. Recently, to further
guarantee readability, more and more works turn to generate sentence-level
human language explanations. However, current works pursuing sentence-level
explanations rely heavily on annotated training data, which limits the
development of interpretability to only a few tasks. As far as we know, this
paper is the first to explore this problem smoothly from weak-supervised
learning to unsupervised learning. Besides, we also notice the high latency of
autoregressive sentence-level explanation generation, which leads to
asynchronous interpretability after prediction. Therefore, we propose a
non-autoregressive interpretable model to facilitate parallel explanation
generation and simultaneous prediction. Through extensive experiments on
Natural Language Inference task and Spouse Prediction task, we find that users
are able to train classifiers with comparable performance $10-15\times$ faster
with parallel explanation generation using only a few or no annotated training
data.

本文旨在探讨如何更好地提供可解释的语言模型预测，通过将训练数据从强监督学习转向弱监督和无监督学习，并提出一种既实现并行解释生成和同时预测的非自回归解释模型，能够以比目前自回归模型快 10-15 倍的速度训练分类器。