The study of emergent communication has been dedicated to interactive
artificial intelligence. While existing work focuses on communication about
single objects or complex image scenes, we argue that communicating
relationships between multiple objects is important in more realistic tasks,
but understudied. In this paper, we try to fill this gap and focus on emergent
communication about positional relationships between two objects. We train
agents in the referential game where observations contain two objects, and find
that generalization is the major problem when the positional relationship is
involved. The key factor affecting the generalization ability of the emergent
language is the input variation between Speaker and Listener, which is realized
by a random image generator in our work. Further, we find that the learned
language can generalize well in a new multi-step MDP task where the positional
relationship describes the goal, and performs better than raw-pixel images as
well as pre-trained image features, verifying the strong generalization ability
of discrete sequences. We also show that language transfer from the referential
game performs better in the new task than learning language directly in this
task, implying the potential benefits of pre-training in referential games. All
in all, our experiments demonstrate the viability and merit of having agents
learn to communicate positional relationships between multiple objects through
emergent communication.

本文探讨基于 emergent communication 的多物体位置关系引用游戏中语言的学习，发现输入变异性对语义广义能力有重要影响，并通过 MDP 任务验证了学到语言对未知场景的泛化能力，同时表明了学习上下文相关语言的潜在优势。