Figurative language understanding has been recently framed as a recognizing
textual entailment (RTE) task (a.k.a. natural language inference, or NLI).
However, similar to classical RTE/NLI datasets, the current benchmarks suffer
from spurious correlations and annotation artifacts. To tackle this problem,
work on NLI has built explanation-based datasets such as e-SNLI, allowing us to
probe whether language models are right for the right reasons.Yet no such data
exists for figurative language, making it harder to assess genuine
understanding of such expressions. To address this issue, we release FLUTE, a
dataset of 9,000 figurative NLI instances with explanations, spanning four
categories: Sarcasm, Simile, Metaphor, and Idioms. We collect the data through
a model-in-the-loop framework based on GPT-3, crowd workers, and expert
annotators. We show how utilizing GPT-3 in conjunction with human annotators
(novices and experts) can aid in scaling up the creation of datasets even for
such complex linguistic phenomena as figurative language. The baseline
performance of the T5 model fine-tuned on FLUTE shows that our dataset can
bring us a step closer to developing models that understand figurative language
through textual explanations.

FLUTE 是一个包含 9000 个各类比喻语言例子的数据集，使用 GPT-3、众包工人和专家注释人员构建。在该研究中，T5 模型 fine-tuned 后的基准性能表明，我们的数据集通过文本解释可以让我们更好地理解比喻语言。

FLUTE：通过文本解释理解比喻语言

FLUTE: Figurative Language Understanding through Textual Explanations

In order for machine learning to garner widespread public adoption, models
must be able to provide interpretable and robust explanations for their
decisions, as well as learn from human-provided explanations at train time. In
this work, we extend the Stanford Natural Language Inference dataset with an
additional layer of human-annotated natural language explanations of the
entailment relations. We further implement models that incorporate these
explanations into their training process and output them at test time. We show
how our corpus of explanations, which we call e-SNLI, can be used for various
goals, such as obtaining full sentence justifications of a model's decisions,
improving universal sentence representations and transferring to out-of-domain
NLI datasets. Our dataset thus opens up a range of research directions for
using natural language explanations, both for improving models and for
asserting their trust.

通过扩展自然语言推理数据集并引入人工提供的自然语言解释，我们实现了能够学习人类提供解释的机器学习模型，并将其用于提高模型性能和确定其可靠性的研究。