Shortcut reasoning is an irrational process of inference, which degrades the
robustness of an NLP model. While a number of previous work has tackled the
identification of shortcut reasoning, there are still two major limitations:
(i) a method for quantifying the severity of the discovered shortcut reasoning
is not provided; (ii) certain types of shortcut reasoning may be missed. To
address these issues, we propose a novel method for identifying shortcut
reasoning. The proposed method quantifies the severity of the shortcut
reasoning by leveraging out-of-distribution data and does not make any
assumptions about the type of tokens triggering the shortcut reasoning. Our
experiments on Natural Language Inference and Sentiment Analysis demonstrate
that our framework successfully discovers known and unknown shortcut reasoning
in the previous work.

通过使用超出分布数据的方法，我们提出了一种识别快速推理的新方法，该方法不仅可以量化快速推理的严重程度，还不需要假设触发快速推理的令牌类型。我们在自然语言推理和情感分析方面的实验中成功地发现了之前研究中已知和未知的快速推理。