We present a computational model for a symbol emergence system that enables
the emergence of lexical knowledge with combinatoriality among agents through a
Metropolis-Hastings naming game and cross-situational learning. Many
computational models have been proposed to investigate combinatoriality in
emergent communication and symbol emergence in cognitive and developmental
robotics. However, existing models do not sufficiently address category
formation based on sensory-motor information and semiotic communication through
the exchange of word sequences within a single integrated model. Our proposed
model facilitates the emergence of lexical knowledge with combinatoriality by
performing category formation using multimodal sensory-motor information and
enabling semiotic communication through the exchange of word sequences among
agents in a unified model. Furthermore, the model enables an agent to predict
sensory-motor information for unobserved situations by combining words
associated with categories in each modality. We conducted two experiments with
two humanoid robots in a simulated environment to evaluate our proposed model.
The results demonstrated that the agents can acquire lexical knowledge with
combinatoriality through interpersonal cross-situational learning based on the
Metropolis-Hastings naming game and cross-situational learning. Furthermore,
our results indicate that the lexical knowledge developed using our proposed
model exhibits generalization performance for novel situations through
interpersonal cross-modal inference.

本文提出了一种计算模型，该模型通过 Metropolis-Hastings 命名游戏和跨情景学习，使代理之间的组合性出现，并使语词知识生成，从而实现了类别形成和交际式通信。

符号出现作为人际情境跨越学习：组合词汇知识的出现

Symbol emergence as interpersonal cross-situational learning: the  emergence of lexical knowledge with combinatoriality

In this study, we explore the emergence of symbols during interactions
between individuals through an experimental semiotic study. Previous studies
investigate how humans organize symbol systems through communication using
artificially designed subjective experiments. In this study, we have focused on
a joint attention-naming game (JA-NG) in which participants independently
categorize objects and assign names while assuming their joint attention.
In the theory of the Metropolis-Hastings naming game (MHNG), listeners accept
provided names according to the acceptance probability computed using the
Metropolis-Hastings (MH) algorithm. The theory of MHNG suggests that symbols
emerge as an approximate decentralized Bayesian inference of signs, which is
represented as a shared prior variable if the conditions of MHNG are satisfied.
This study examines whether human participants exhibit behavior consistent
with MHNG theory when playing JA-NG. By comparing human acceptance decisions of
a partner's naming with acceptance probabilities computed in the MHNG, we
tested whether human behavior is consistent with the MHNG theory. The main
contributions of this study are twofold. First, we reject the null hypothesis
that humans make acceptance judgments with a constant probability, regardless
of the acceptance probability calculated by the MH algorithm. This result
suggests that people followed the acceptance probability computed by the MH
algorithm to some extent. Second, the MH-based model predicted human
acceptance/rejection behavior more accurately than the other four models:
Constant, Numerator, Subtraction, and Binary. This result indicates that symbol
emergence in JA-NG can be explained using MHNG and is considered an approximate
decentralized Bayesian inference.

研究人员通过实验符号学研究探索个体之间交互中符号的出现。该研究对主动注意并称呼游戏进行了研究，并比较了人类接受概率和通过 Metropolis-Hastings 算法计算的接受概率的差异。研究表明，在 JA-NG 下进行符号出现的 MHNG 模型可以解释人类的行为，并认为这是一种近似的分散式贝叶斯推断。

联合注意命名博弈中的 Metropolis-Hastings 算法：实验符号学研究

Metropolis-Hastings algorithm in joint-attention naming game:  Experimental semiotics study

Constructive studies on symbol emergence systems seek to investigate
computational models that can better explain human language evolution, the
creation of symbol systems, and the construction of internal representations.
This study provides a new model for emergent communication, which is based on a
probabilistic generative model (PGM) instead of a discriminative model based on
deep reinforcement learning. We define the Metropolis-Hastings (MH) naming game
by generalizing previously proposed models. It is not a referential game with
explicit feedback, as assumed by many emergent communication studies. Instead,
it is a game based on joint attention without explicit feedback.
Mathematically, the MH naming game is proved to be a type of MH algorithm for
an integrative PGM that combines two agents that play the naming game. From
this viewpoint, symbol emergence is regarded as decentralized Bayesian
inference, and semiotic communication is regarded as inter-personal cross-modal
inference. This notion leads to the collective predictive coding hypothesis}
regarding language evolution and, in general, the emergence of symbols. We also
propose the inter-Gaussian mixture model (GMM)+ variational autoencoder (VAE),
a deep generative model for emergent communication based on the MH naming game.
The model has been validated on MNIST and Fruits 360 datasets. Experimental
findings demonstrate that categories are formed from real images observed by
agents, and signs are correctly shared across agents by successfully utilizing
both of the observations of agents via the MH naming game. Furthermore,
scholars verified that visual images were recalled from signs uttered by
agents. Notably, emergent communication without supervision and reward feedback
improved the performance of the unsupervised representation learning of agents.

该研究探索了一种新的通过基于概率生成模型的命名游戏来实现分布式贝叶斯推断的符号产生系统，并提出了利用无监督学习的 Emergent communication 方式来改善代理人的表现。