Decisions of complex language understanding models can be rationalized by limiting their inputs to a relevant subsequence of the original text. A rationale should be as concise as possible without significantly degrading task performance, but this balance can be difficult to achieve in practice. In this paper, we show that it is possible to better manage this trade-off by optimizing a bound on the Information Bottleneck (IB) objective. Our fully unsupervised approach jointly learns an explainer that predicts sparse binary masks over sentences, and an end-task predictor that considers only the extracted rationale. Using IB, we derive a learning objective that allows direct control of mask sparsity levels through a tunable sparse prior. Experiments on ERASER benchmark tasks demonstrate significant gains over norm-minimization techniques for both task performance and agreement with human rationales. Furthermore, we find that in the semi-supervised setting, a modest amount of gold rationales (25% of training examples) closes the gap with a model that uses the full input. Code: https://github.com/bhargaviparanjape/explainable_qa

本文通过优化信息瓶颈（IB）目标的界限，提出了一种新的完全无监督方法， jointly 学习解释器和端任务预测器，通过可调的稀疏先验直接控制掩码的稀疏性水平。在 ERASER 基准任务上的实验表明，使用 IB 相比于 norm-minimization 技术，在任务性能和与人类理性的一致性方面都取得了显著的收益。此外，我们发现在半监督学习的情况下，对于训练样本的25％的黄金理性，能够弥补使用完整输入的模型的差距。

信息瓶颈方法控制合理化提取中的简洁性