There is growing interest in studying the languages that emerge when neural
agents are jointly trained to solve tasks requiring communication through a
discrete channel. We investigate here the information-theoretic complexity of
such languages, focusing on the basic two-agent, one-exchange setup. We find
that, under common training procedures, the emergent languages are subject to
an entropy minimization pressure that has also been detected in human language,
whereby the mutual information between the communicating agent's inputs and the
messages is minimized, within the range afforded by the need for successful
communication. That is, emergent languages are (nearly) as simple as the task
they are developed for allow them to be. This pressure is amplified as we
increase communication channel discreteness. Further, we observe that stronger
discrete-channel-driven entropy minimization leads to representations with
increased robustness to overfitting and adversarial attacks. We conclude by
discussing the implications of our findings for the study of natural and
artificial communication systems.

研究神经代理在通讯任务中共同训练时出现的语言，并聚焦于两个代理一次交流的基本设置，发现这类语言呈现出信息论复杂性且受到熵极小化压力的影响。