In recent years, several machine learning models have been proposed. They are
trained with a language modelling objective on large-scale text-only data. With
such pretraining, they can achieve impressive results on many Natural Language
Understanding and Generation tasks. However, many facets of meaning cannot be
learned by ``listening to the radio" only. In the literature, many
Vision+Language (V+L) tasks have been defined with the aim of creating models
that can ground symbols in the visual modality. In this work, we provide a
systematic literature review of several tasks and models proposed in the V+L
field. We rely on Wittgenstein's idea of `language games' to categorise such
tasks into 3 different families: 1) discriminative games, 2) generative games,
and 3) interactive games. Our analysis of the literature provides evidence that
future work should be focusing on interactive games where communication in
Natural Language is important to resolve ambiguities about object referents and
action plans and that physical embodiment is essential to understand the
semantics of situations and events. Overall, these represent key requirements
for developing grounded meanings in neural models.

对于基于语言模型目标在大规模纯文本数据上进行训练的几种机器学习模型，在许多自然语言理解和生成任务上取得了令人印象深刻的结果。然而，对于语义的许多方面，仅仅通过 “倾听收音机” 是无法学习到的。本文系统地回顾了视觉 + 语言领域中提出的多个任务和模型。利用维特根斯坦的 “语言游戏” 思想，将这些任务分为 3 个不同的类别：1）区分性游戏，2）生成性游戏，和 3）交互式游戏。我们的文献分析提供了证据，表明未来的工作应该专注于重要的交互式游戏，在其中自然语言的交流对于解决关于物体指称和行动计划的不确定性是必要的，而物理体现则是理解场景和事件语义的重要条件。作为总体，这些是发展基于神经模型的扎根意义的关键要求。