Humans can quickly learn a new word from a few illustrative examples, and then systematically and flexibly use it in novel contexts. Yet the abilities of current language models for few-shot word learning, and methods for improving these abilities, are underexplored. In this study, we introduce a novel method, Meta-training for IN-context learNing Of Words (Minnow). This method trains language models to generate new examples of a word's usage given a few in-context examples, using a special placeholder token to represent the new word. This training is repeated on many new words to develop a general word-learning ability. We find that training models from scratch with Minnow on human-scale child-directed language enables strong few-shot word learning, comparable to a large language model (LLM) pre-trained on orders of magnitude more data. Furthermore, through discriminative and generative evaluations, we demonstrate that finetuning pre-trained LLMs with Minnow improves their ability to discriminate between new words, identify syntactic categories of new words, and generate reasonable new usages and definitions for new words, based on one or a few in-context examples. These findings highlight the data efficiency of Minnow and its potential to improve language model performance in word learning tasks.

本研究针对当前语言模型在少量例子下新词学习能力不足的问题，提出了一种新方法——“元训练上下文学习单词”（Minnow）。研究表明，使用该方法从人类儿童导向语言中训练模型，可以在少量示例下实现强大的新词学习能力，显著提高预训练大语言模型的表现，展现了其在词汇学习任务中的高数据效率和潜在应用价值。

通过元上下文学习的快速词汇学习