We relax the constraint of a shared language between agents in a semantic and
goal-oriented communication system to explore the effect of language mismatch
in distributed task solving. We propose a mathematical framework, which
provides a modelling and a measure of the semantic distortion introduced in the
communication when agents use distinct languages. We then propose a new
approach to semantic channel equalization with proven effectiveness through
numerical evaluations.

我们通过一个语义和目标导向的通信系统，放宽了代理之间的共享语言的约束，以探索在分布式任务求解中语言不匹配的影响。我们提出了一个数学框架，模拟并度量了代理使用不同语言时引入的语义失真，并通过数值评估证明了一种新的语义通道均衡方法的有效性。

语义通道均衡的潜在空间对齐

Latent Space Alignment for Semantic Channel Equalization

In this paper, we investigate whether artificial agents can develop a shared
language in an ecological setting where communication relies on a sensory-motor
channel. To this end, we introduce the Graphical Referential Game (GREG) where
a speaker must produce a graphical utterance to name a visual referent object
while a listener has to select the corresponding object among distractor
referents, given the delivered message. The utterances are drawing images
produced using dynamical motor primitives combined with a sketching library. To
tackle GREG we present CURVES: a multimodal contrastive deep learning mechanism
that represents the energy (alignment) between named referents and utterances
generated through gradient ascent on the learned energy landscape. We
demonstrate that CURVES not only succeeds at solving the GREG but also enables
agents to self-organize a language that generalizes to feature compositions
never seen during training. In addition to evaluating the communication
performance of our approach, we also explore the structure of the emerging
language. Specifically, we show that the resulting language forms a coherent
lexicon shared between agents and that basic compositional rules on the
graphical productions could not explain the compositional generalization.

本文研究了人工智能代理程序是否能够在基于感觉 - 运动通道的生态环境中发展出共享语言；介绍了图形指涉游戏（GREG）以及 CURVES：一种多模式对比深度学习机制，监测了 CURVES 在训练中的表现以及代理程序能否自组织出通用于未见过情况下的特征组合的语言；同时，探讨了新兴的语言结构。