Recent advances in large language models using deep learning techniques have
renewed interest on how languages can be learned from data. However, it is
unclear whether or how these models represent grammatical information from the
learned languages. In addition, the models must be pre-trained on large corpora
before they can be used. In this work, we propose an alternative, more
transparent and cognitively plausible architecture for learning language.
Instead of using deep learning, our approach uses a minimal cognitive
architecture based on sequence memory and chunking. The learning mechanism is
based on the principles of reinforcement learning. We test our architecture on
a number of natural-like toy languages. Results show that the model can learn
these artificial languages from scratch and extract grammatical information
that supports learning. Our study demonstrates the power of this simple
architecture and stresses the importance of sequence memory as a key component
of the language learning process. Since other animals do not seem to have a
faithful sequence memory, this may explain why only humans have developed
complex languages.

通过序列记忆和分块构建的最小认知体系架构用于学习语言，替代了使用深度学习的大型语言模型，并且能够从零开始学习人工语言，并提取支持学习的语法信息。研究表明这种简单的架构的强大性，并强调序列记忆作为语言学习过程的关键组成部分的重要性，这可能解释了仅人类发展了复杂语言的原因。

揭开语言习得的黑盒子

Opening the black box of language acquisition

Sentence embeddings induced with various transformer architectures encode
much semantic and syntactic information in a distributed manner in a
one-dimensional array. We investigate whether specific grammatical information
can be accessed in these distributed representations. Using data from a task
developed to test rule-like generalizations, our experiments on detecting
subject-verb agreement yield several promising results. First, we show that
while the usual sentence representations encoded as one-dimensional arrays do
not easily support extraction of rule-like regularities, a two-dimensional
reshaping of these vectors allows various learning architectures to access such
information. Next, we show that various architectures can detect patterns in
these two-dimensional reshaped sentence embeddings and successfully learn a
model based on smaller amounts of simpler training data, which performs well on
more complex test data. This indicates that current sentence embeddings contain
information that is regularly distributed, and which can be captured when the
embeddings are reshaped into higher dimensional arrays. Our results cast light
on representations produced by language models and help move towards developing
few-shot learning approaches.

通过使用来自语言模型的更高维数组，我们发现句子嵌入可捕获规则类的语法信息，进而帮助发展少样本学习方法。