Capturing semantic relations between sentences, such as entailment, is a
long-standing challenge for computational semantics. Logic-based models analyse
entailment in terms of possible worlds (interpretations, or situations) where a
premise P entails a hypothesis H iff in all worlds where P is true, H is also
true. Statistical models view this relationship probabilistically, addressing
it in terms of whether a human would likely infer H from P. In this paper, we
wish to bridge these two perspectives, by arguing for a visually-grounded
version of the Textual Entailment task. Specifically, we ask whether models can
perform better if, in addition to P and H, there is also an image
(corresponding to the relevant "world" or "situation"). We use a multimodal
version of the SNLI dataset (Bowman et al., 2015) and we compare "blind" and
visually-augmented models of textual entailment. We show that visual
information is beneficial, but we also conduct an in-depth error analysis that
reveals that current multimodal models are not performing "grounding" in an
optimal fashion.

本文研究了文本蕴涵中的语义关系 —— 通过可能世界来分析前提和假说之间的关系，同时结合图像进行多模态的分析，结果表明文本和视觉信息相结合可以更好地进行文本蕴涵，但当前多模态模型在 “接地” 方面还不够优化。