State-of-the-art NLP methods achieve human-like performance on many tasks,
but make errors nevertheless. Characterizing these errors in easily
interpretable terms gives insight into whether a classifier is prone to making
systematic errors, but also gives a way to act and improve the classifier. We
propose to discover those patterns of tokens that distinguish correct and
erroneous predictions as to obtain global and interpretable descriptions for
arbitrary NLP classifiers. We formulate the problem of finding a succinct and
non-redundant set of such patterns in terms of the Minimum Description Length
principle. Through an extensive set of experiments, we show that our method,
Premise, performs well in practice. Unlike existing solutions, it recovers
ground truth, even on highly imbalanced data over large vocabularies. In VQA
and NER case studies, we confirm that it gives clear and actionable insight
into the systematic errors made by NLP classifiers.

我们提出了一种方法，通过发现那些区分正确和错误预测的标记模式来获得全局且可解释的描述，从而改善 NLP 分类器的性能。通过一系列实验证明，我们的方法在实践中表现良好，能够在大词汇量的不平衡数据上恢复出真实情况，并对 NLP 分类器的系统错误提供明确可行的见解。

通过可解释的标记模式理解和减轻分类错误

Understanding and Mitigating Classification Errors Through Interpretable  Token Patterns

Neural networks lack the ability to reason about qualitative physics and so
cannot generalize to scenarios and tasks unseen during training. We propose
ESPRIT, a framework for commonsense reasoning about qualitative physics in
natural language that generates interpretable descriptions of physical events.
We use a two-step approach of first identifying the pivotal physical events in
an environment and then generating natural language descriptions of those
events using a data-to-text approach. Our framework learns to generate
explanations of how the physical simulation will causally evolve so that an
agent or a human can easily reason about a solution using those interpretable
descriptions. Human evaluations indicate that ESPRIT produces crucial
fine-grained details and has high coverage of physical concepts compared to
even human annotations. Dataset, code and documentation are available at
this https URL

提出了一种用于定性物理常识推理的 ESPRTI 框架，可以使用解释性自然语言描述物理事件，从而灵活地适应未被训练过的场景和任务。