Recently, an increasing number of works have introduced models capable of generating natural language explanations (NLEs) for their predictions on vision-language (VL) tasks. Such models are appealing because they can provide human-friendly and comprehensive explanations. However, there is still a lack of unified evaluation approaches for the explanations generated by these models. Moreover, there are currently only few datasets of NLEs for VL tasks. In this work, we introduce e-ViL, a benchmark for explainable vision-language tasks that establishes a unified evaluation framework and provides the first comprehensive comparison of existing approaches that generate NLEs for VL tasks. e-ViL spans four models and three datasets. Both automatic metrics and human evaluation are used to assess model-generated explanations. We also introduce e-SNLI-VE, the largest existing VL dataset with NLEs (over 430k instances). Finally, we propose a new model that combines UNITER, which learns joint embeddings of images and text, and GPT-2, a pre-trained language model that is well-suited for text generation. It surpasses the previous state-of-the-art by a large margin across all datasets.

本研究介绍了 e-ViL 和 e-SNLI-VE 为可解释的视觉语言任务建立了一个统一的评估框架。该任务旨在生成自然语言解释，并涉及四个模型和三个数据集。研究人员还提出了一种新模型，该模型结合了 UNITER 和 GPT-2，对所有数据集的现有技术水平取得了很大的突破。

e-ViL: 视觉语言任务中自然语言解释的数据集和基准测试