A fundamental characteristic common to both human vision and natural language
is their compositional nature. Yet, despite the performance gains contributed
by large vision and language pretraining, recent investigations find that
most-if not all-our state-of-the-art vision-language models struggle at
compositionality. They are unable to distinguish between images of " a girl in
white facing a man in black" and "a girl in black facing a man in white".
Moreover, prior work suggests that compositionality doesn't arise with scale:
larger model sizes or training data don't help. This paper develops a new
iterated training algorithm that incentivizes compositionality. We draw on
decades of cognitive science research that identifies cultural transmission-the
need to teach a new generation-as a necessary inductive prior that incentivizes
humans to develop compositional languages. Specifically, we reframe
vision-language contrastive learning as the Lewis Signaling Game between a
vision agent and a language agent, and operationalize cultural transmission by
iteratively resetting one of the agent's weights during training. After every
iteration, this training paradigm induces representations that become "easier
to learn", a property of compositional languages: e.g. our model trained on
CC3M and CC12M improves standard CLIP by 4.7%, 4.0% respectfully in the
SugarCrepe benchmark.

通过引入一种新的迭代训练算法，以文化传播作为归纳先验，本研究以视觉代理和语言代理之间的 Lewis 信号游戏重新定义了视觉 - 语言对比学习，并通过在训练过程中迭代重置代理权重的方式实现文化传播，使得模型对于合成语言的特性变得更易于学习，达到了在 SugarCrepe 评测中相比标准 CLIP 模型分别提升 4.7% 和 4.0% 的效果。

迭代学习在大型视觉语言模型中提高组合性

Iterated Learning Improves Compositionality in Large Vision-Language  Models

Cultural transmission is the domain-general social skill that allows agents
to acquire and use information from each other in real-time with high fidelity
and recall. In humans, it is the inheritance process that powers cumulative
cultural evolution, expanding our skills, tools and knowledge across
generations. We provide a method for generating zero-shot, high recall cultural
transmission in artificially intelligent agents. Our agents succeed at
real-time cultural transmission from humans in novel contexts without using any
pre-collected human data. We identify a surprisingly simple set of ingredients
sufficient for generating cultural transmission and develop an evaluation
methodology for rigorously assessing it. This paves the way for cultural
evolution as an algorithm for developing artificial general intelligence.

该研究提供了一种为人工智能代理产生零样本、高召回文化传播的方法，并鉴别了一组简单的材料，以生成文化传播，并开发了一种严格评估文化传播的方法，为文化进化成为开发人工智能的算法铺平了道路。

学习在没有人类数据的情况下实现鲁棒的实时文化传播

Learning Robust Real-Time Cultural Transmission without Human Data

Referential games offer a grounded learning environment for neural agents
which accounts for the fact that language is functionally used to communicate.
However, they do not take into account a second constraint considered to be
fundamental for the shape of human language: that it must be learnable by new
language learners.
Cogswell et al. (2019) introduced cultural transmission within referential
games through a changing population of agents to constrain the emerging
language to be learnable. However, the resulting languages remain inherently
biased by the agents' underlying capabilities.
In this work, we introduce Language Transmission Engine to model both
cultural and architectural evolution in a population of agents. As our core
contribution, we empirically show that the optimal situation is to take into
account also the learning biases of the language learners and thus let language
and agents co-evolve. When we allow the agent population to evolve through
architectural evolution, we achieve across the board improvements on all
considered metrics and surpass the gains made with cultural transmission. These
results stress the importance of studying the underlying agent architecture and
pave the way to investigate the co-evolution of language and agent in language
emergence studies.

通过模拟代理人群体的文化和架构演进，研究发现，当语言习得者的学习偏好被考虑在内时，语言和代理人的共同进化可以得到最优解，这一发现强调了研究代理人架构的重要性，并为语言的起源研究铺平了道路。