Guessing games are a prototypical instance of the "learning by interacting" paradigm. This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA). We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL). We evaluate the ability of both procedures to generalize: an in-domain evaluation shows an increased accuracy (+7.79) compared with competitors on the evaluation suite CompGuessWhat?!; a transfer evaluation shows improved performance for VQA on the TDIUC dataset in terms of harmonic average accuracy (+5.31) thanks to more fine-grained object representations learned via SPIEL.

本文研究了一个人工智能代理在玩猜谜游戏时如何受益，以便之后在诸如视觉问答（VQA）之类的新领域中表现更好，并提出了两种方法来利用猜谜游戏：一种是监督学习场景，代理学习模仿成功的猜测游戏，另一种是自我游戏，称为自我游戏通过迭代经验学习（SPIEL）。作者评估了两个过程的泛化能力，并在CompGuessWhat？！测评工具套件上展示了评估结果。结果显示，在范围内评估中，与竞争对手相比，代理的准确率提高了7.79个点；在转移评估中，得益于通过SPIEL学习的更细粒度的对象表示，VQA在TDIUC数据集上的表现在谐波平均准确度上提高了5.31个点。

通过视觉猜谜游戏学习到的神经表示的泛化能力实证研究