A promising approach for teaching artificial agents to use natural language
involves using human-in-the-loop training. However, recent work suggests that
current machine learning methods are too data inefficient to be trained in this
way from scratch. In this paper, we investigate the relationship between two
categories of learning signals with the ultimate goal of improving sample
efficiency: imitating human language data via supervised learning, and
maximizing reward in a simulated multi-agent environment via self-play (as done
in emergent communication), and introduce the term supervised self-play (S2P)
for algorithms using both of these signals. We find that first training agents
via supervised learning on human data followed by self-play outperforms the
converse, suggesting that it is not beneficial to emerge languages from
scratch. We then empirically investigate various S2P schedules that begin with
supervised learning in two environments: a Lewis signaling game with symbolic
inputs, and an image-based referential game with natural language descriptions.
Lastly, we introduce population based approaches to S2P, which further improves
the performance over single-agent methods.

本研究探讨了一种新的教授人工智能代理人使用自然语言的方法，发现在通过受监督的学习奖励和自我对弈的信号两种方式的训练后比直接从头开始进行自我学习效果更好，并提出了一种基于人群的方法以进一步提高表现。