Metaphor identification aims at understanding whether a given expression is
used figuratively in context. However, in this paper we show how existing
metaphor identification datasets can be gamed by fully ignoring the potential
metaphorical expression or the context in which it occurs. We test this
hypothesis in a variety of datasets and settings, and show that metaphor
identification systems based on language models without complete information
can be competitive with those using the full context. This is due to the
construction procedures to build such datasets, which introduce unwanted biases
for positive and negative classes. Finally, we test the same hypothesis on
datasets that are carefully sampled from natural corpora and where this bias is
not present, making these datasets more challenging and reliable.

通过忽略潜在的比喻表达或其出现的上下文，本文展示了现有的比喻识别数据集如何被篡改。我们在各种数据集和环境中测试了这一假设，并表明基于语言模型的比喻识别系统在没有完整信息的情况下与使用完整上下文信息的系统具有竞争力，这是由于构建此类数据集的过程引入了对正面和负面类别的不希望的偏倚。最后，我们在从自然语料库中精心抽样且没有存在该偏倚的数据集上测试了同样的假设，使得这些数据集更具挑战性和可靠性。