Human language acquisition is an efficient, supervised, and continual
process. In this work, we took inspiration from how human babies acquire their
first language, and developed a computational process for word acquisition
through comparative learning. Motivated by cognitive findings, we generated a
small dataset that enables the computation models to compare the similarities
and differences of various attributes, learn to filter out and extract the
common information for each shared linguistic label. We frame the acquisition
of words as not only the information filtration process, but also as
representation-symbol mapping. This procedure does not involve a fixed
vocabulary size, nor a discriminative objective, and allows the models to
continually learn more concepts efficiently. Our results in controlled
experiments have shown the potential of this approach for efficient continual
learning of grounded words.

通过比较学习的计算过程，本研究模拟了人类婴儿习得第一门语言的方式，开发了一种用于词汇习得的计算过程，通过筛选和提取共享语言标签的共同信息，并将词汇的习得过程视为信息过滤和符号映射，以实现有效的持续学习。在对照实验中得到的结果显示了这种方法在词汇习得方面的潜力。

人类启发的渐进对齐与比较学习 用于基于语境的词汇习得

Human Inspired Progressive Alignment and Comparative Learning for  Grounded Word Acquisition

We introduce a new in-context learning paradigm to measure Large Language
Models' (LLMs) ability to learn novel words during inference. In particular, we
rewrite Winograd-style co-reference resolution problems by replacing the key
concept word with a synthetic but plausible word that the model must understand
to complete the task. Solving this task requires the model to make use of the
dictionary definition of the new word given in the prompt. This benchmark
addresses word acquisition, one important aspect of the diachronic degradation
known to afflict LLMs. As LLMs are frozen in time at the moment they are
trained, they are normally unable to reflect the way language changes over
time. We show that the accuracy of LLMs compared to the original Winograd tasks
decreases radically in our benchmark, thus identifying a limitation of current
models and providing a benchmark to measure future improvements in LLMs ability
to do in-context learning.

本文介绍了一种新的上下文学习范例，以测量大型语言模型在推理过程中学习新单词的能力，并通过 Winograd 风格的共指解决问题提出了基准测试，探讨了 LLMs 的时序退化现象和其限制。

WinoDict: 探究语言模型的上下文词汇习得能力

WinoDict: Probing language models for in-context word acquisition

We investigate how neural language models acquire individual words during
training, extracting learning curves and ages of acquisition for over 600 words
on the MacArthur-Bates Communicative Development Inventory (Fenson et al.,
2007). Drawing on studies of word acquisition in children, we evaluate multiple
predictors for words' ages of acquisition in LSTMs, BERT, and GPT-2. We find
that the effects of concreteness, word length, and lexical class are pointedly
different in children and language models, reinforcing the importance of
interaction and sensorimotor experience in child language acquisition. Language
models rely far more on word frequency than children, but like children, they
exhibit slower learning of words in longer utterances. Interestingly, models
follow consistent patterns during training for both unidirectional and
bidirectional models, and for both LSTM and Transformer architectures. Models
predict based on unigram token frequencies early in training, before
transitioning loosely to bigram probabilities, eventually converging on more
nuanced predictions. These results shed light on the role of distributional
learning mechanisms in children, while also providing insights for more
human-like language acquisition in language models.

研究神经语言模型在训练过程中如何获取单词，并提取 MacArthur-Bates 交际发展清单上 600 多个单词的学习曲线和获取年龄。与儿童的单词获取研究结合，评估 LSTMs、BERT 和 GPT-2 模型的多种预测因素，发现模型对单词频率的依赖程度远高于儿童，但与儿童相似的是，它们在更长的话语中学习单词较慢。同时，模型在训练过程中遵循一致的模式，这些结果为人类语言模型的更类人化语言习得提供了启示。