While multi-agent reinforcement learning has been used as an effective means
to study emergent communication between agents, existing work has focused
almost exclusively on communication with discrete symbols. Human communication
often takes place (and emerged) over a continuous acoustic channel; human
infants acquire language in large part through continuous signalling with their
caregivers. We therefore ask: Are we able to observe emergent language between
agents with a continuous communication channel trained through reinforcement
learning? And if so, what is the impact of channel characteristics on the
emerging language? We propose an environment and training methodology to serve
as a means to carry out an initial exploration of these questions. We use a
simple messaging environment where a "speaker" agent needs to convey a concept
to a "listener". The Speaker is equipped with a vocoder that maps symbols to a
continuous waveform, this is passed over a lossy continuous channel, and the
Listener needs to map the continuous signal to the concept. Using deep
Q-learning, we show that basic compositionality emerges in the learned language
representations. We find that noise is essential in the communication channel
when conveying unseen concept combinations. And we show that we can ground the
emergent communication by introducing a caregiver predisposed to "hearing" or
"speaking" English. Finally, we describe how our platform serves as a starting
point for future work that uses a combination of deep reinforcement learning
and multi-agent systems to study our questions of continuous signalling in
language learning and emergence.

通过使用深度 Q 学习，研究了多智能体强化学习在具有连续通信通道时的应用，发现基本组合特性会在学习的语言表示中出现。噪声在传达未经历过的概念组合时很重要，引入有倾向性的护理者可以使得新的交流创造出一种更有意义的基础语言。这个研究为深度强化学习与多智能体系统的进一步相关研究提供了平台。