Humans can learn languages from remarkably little experience. Developing
computational models that explain this ability has been a major challenge in
cognitive science. Bayesian models that build in strong inductive biases -
factors that guide generalization - have been successful at explaining how
humans might generalize from few examples in controlled settings but are
usually too restrictive to be tractably applied to more naturalistic data. By
contrast, neural networks have flexible representations that allow them to
learn well from naturalistic data but require many more examples than humans
receive. We show that learning from limited naturalistic data is possible with
an approach that combines the strong inductive biases of a Bayesian model with
the flexible representations of a neural network. This approach works by
distilling a Bayesian model's biases into a neural network. Like a Bayesian
model, the resulting system can learn formal linguistic patterns from a small
number of examples. Like a neural network, it can also learn aspects of English
syntax from a corpus of natural language - and it outperforms a standard neural
network at acquiring the linguistic phenomena of recursion and priming.
Bridging the divide between Bayesian models and neural networks makes it
possible to handle a broader range of learning scenarios than either approach
can handle on its own.

将贝叶斯模型的归纳偏差与神经网络的灵活表示相结合，使得从自然语素材中有限学习成为可能。

将贝叶斯先验蒸馏到人工神经网络中，模拟快速语言学习

Modeling rapid language learning by distilling Bayesian priors into  artificial neural networks

Recent advances in self-supervised modeling of text and images open new
opportunities for computational models of child language acquisition, which is
believed to rely heavily on cross-modal signals. However, prior studies have
been limited by their reliance on vision models trained on large image datasets
annotated with a pre-defined set of depicted object categories. This is (a) not
faithful to the information children receive and (b) prohibits the evaluation
of such models with respect to category learning tasks, due to the pre-imposed
category structure. We address this gap, and present a cognitively-inspired,
multimodal acquisition model, trained from image-caption pairs on naturalistic
data using cross-modal self-supervision. We show that the model learns word
categories and object recognition abilities, and presents trends reminiscent of
those reported in the developmental literature. We make our code and trained
models public for future reference and use.

本文提出一种认知启发的、多模态获取模型，通过跨模态自监督在自然数据上从图像 - 标题对进行训练，展示了该模型学习词汇分类和物体识别能力，并展现了类似于发展心理学文献中报告的趋势。我们公开了我们的代码和训练模型，供未来参考和使用。

多模词汇分类的计算获取模型

A Computational Acquisition Model for Multimodal Word Categorization

Large transformer-based models are able to perform in-context few-shot
learning, without being explicitly trained for it. This observation raises the
question: what aspects of the training regime lead to this emergent behavior?
Here, we show that this behavior is driven by the distributions of the training
data itself. In-context learning emerges when the training data exhibits
particular distributional properties such as burstiness (items appear in
clusters rather than being uniformly distributed over time) and having large
numbers of rarely occurring classes. In-context learning also emerges more
strongly when item meanings or interpretations are dynamic rather than fixed.
These properties are exemplified by natural language, but are also inherent to
naturalistic data in a wide range of other domains. They also depart
significantly from the uniform, i.i.d. training distributions typically used
for standard supervised learning. In our initial experiments, we found that
in-context learning traded off against more conventional weight-based learning,
and models were unable to achieve both simultaneously. However, our later
experiments uncovered that the two modes of learning could co-exist in a single
model when it was trained on data following a skewed Zipfian distribution --
another common property of naturalistic data, including language. In further
experiments, we found that naturalistic data distributions were only able to
elicit in-context learning in transformers, and not in recurrent models. In
sum, our findings indicate how the transformer architecture works together with
particular properties of the training data to drive the intriguing emergent
in-context learning behaviour of large language models, and how future work
might encourage both in-context and in-weights learning in domains beyond
language.

研究了 Transformer-based 模型在 in-context few-shot learning 方面的行为和处理方式，发现训练数据的分布特性是这种行为的关键，特别是数据具有大量稀有类别和爆发性分布这两个属性时；而这些属性往往是自然数据所具有的。此外发现在特定训练数据下使用 Zipfian distribution 可以支持 Transformer-based 模型同时实现 in-context 和 in-weights learning。

数据分布特征驱动变压器模型中的上下文学习

Data Distributional Properties Drive Emergent In-Context Learning in Transformers

Recognizing emotions during social interactions has many potential
applications with the popularization of low-cost mobile sensors, but a
challenge remains with the lack of naturalistic affective interaction data.
Most existing emotion datasets do not support studying idiosyncratic emotions
arising in the wild as they were collected in constrained environments.
Therefore, studying emotions in the context of social interactions requires a
novel dataset, and K-EmoCon is such a multimodal dataset with comprehensive
annotations of continuous emotions during naturalistic conversations. The
dataset contains multimodal measurements, including audiovisual recordings,
EEG, and peripheral physiological signals, acquired with off-the-shelf devices
from 16 sessions of approximately 10-minute long paired debates on a social
issue. Distinct from previous datasets, it includes emotion annotations from
all three available perspectives: self, debate partner, and external observers.
Raters annotated emotional displays at intervals of every 5 seconds while
viewing the debate footage, in terms of arousal-valence and 18 additional
categorical emotions. The resulting K-EmoCon is the first publicly available
emotion dataset accommodating the multiperspective assessment of emotions
during social interactions.

本文介绍了 K-EmoCon 数据集，该数据集支持从多个角度评估社交互动中的持续情感，并包括音频 / 视觉记录、EEG 和外围生理信号等多模态测量。