Figurative language is commonplace in natural language, and while making
communication memorable and creative, can be difficult to understand. In this
work, we investigate the robustness of Question Answering (QA) models on
figurative text. Yes/no questions, in particular, are a useful probe of
figurative language understanding capabilities of large language models. We
propose FigurativeQA, a set of 1000 yes/no questions with figurative and
non-figurative contexts, extracted from the domains of restaurant and product
reviews. We show that state-of-the-art BERT-based QA models exhibit an average
performance drop of up to 15\% points when answering questions from figurative
contexts, as compared to non-figurative ones. While models like GPT-3 and
ChatGPT are better at handling figurative texts, we show that further
performance gains can be achieved by automatically simplifying the figurative
contexts into their non-figurative (literal) counterparts. We find that the
best overall model is ChatGPT with chain-of-thought prompting to generate
non-figurative contexts. Our work provides a promising direction for building
more robust QA models with figurative language understanding capabilities.

通过研究问答模型对比喻文本的鲁棒性，本研究提出了一个包含 1000 个问答问题的数据集 FigurativeQA，展示了当回答含有比喻语境的问题时，基于 BERT 的问答模型相较于非比喻语境会出现性能下降的现象，同时进一步将比喻语境简化为非比喻语境有助于提高模型性能，研究发现基于链式思维提示的 ChatGPT 是最优模型，可以构建具有比喻语言理解能力的更鲁棒的问答模型。