Language models (LMs) have demonstrated remarkable proficiency in generating
linguistically coherent text, sparking discussions about their relevance to
understanding human language learnability. However, a significant gap exists
between the training data for these models and the linguistic input a child
receives. LMs are typically trained on data that is orders of magnitude larger
and fundamentally different from child-directed speech (Warstadt and Bowman,
2022; Warstadt et al., 2023; Frank, 2023a). Addressing this discrepancy, our
research focuses on training LMs on subsets of a single child's linguistic
input. Previously, Wang, Vong, Kim, and Lake (2023) found that LMs trained in
this setting can form syntactic and semantic word clusters and develop
sensitivity to certain linguistic phenomena, but they only considered LSTMs and
simpler neural networks trained from just one single-child dataset. Here, to
examine the robustness of learnability from single-child input, we
systematically train six different model architectures on five datasets (3
single-child and 2 baselines). We find that the models trained on single-child
datasets showed consistent results that matched with previous work,
underscoring the robustness of forming meaningful syntactic and semantic
representations from a subset of a child's linguistic input.

该研究探讨了使用单个儿童语言输入子集训练语言模型，以形成有意义的句法和语义表征，发现这种方法的学习性强度和可靠性。

单个儿童语言输入对可学习性的系统调查

A systematic investigation of learnability from single child linguistic  input

Artificial neural networks can generalize productively to novel contexts. Can
they also learn exceptions to those productive rules? We explore this question
using the case of restrictions on English passivization (e.g., the fact that
"The vacation lasted five days" is grammatical, but "*Five days was lasted by
the vacation" is not). We collect human acceptability judgments for passive
sentences with a range of verbs, and show that the probability distribution
defined by GPT-2, a language model, matches the human judgments with high
correlation. We also show that the relative acceptability of a verb in the
active vs. passive voice is positively correlated with the relative frequency
of its occurrence in those voices. These results provide preliminary support
for the entrenchment hypothesis, according to which learners track and uses the
distributional properties of their input to learn negative exceptions to rules.
At the same time, this hypothesis fails to explain the magnitude of
unpassivizability demonstrated by certain individual verbs, suggesting that
other cues to exceptionality are available in the linguistic input.

本研究探讨了人工神经网络在英语被动语态限制的负面例外的规则学习中的应用

语言模型能够学习语法规则的例外

Language Models Can Learn Exceptions to Syntactic Rules

Neural networks drive the success of natural language processing. A
fundamental property of natural languages is their compositional structure,
allowing us to describe new meanings systematically. However, neural networks
notoriously struggle with systematic generalization and do not necessarily
benefit from a compositional structure in emergent communication simulations.
Here, we test how neural networks compare to humans in learning and
generalizing a new language. We do this by closely replicating an artificial
language learning study (conducted originally with human participants) and
evaluating the memorization and generalization capabilities of deep neural
networks with respect to the degree of structure in the input language. Our
results show striking similarities between humans and deep neural networks:
More structured linguistic input leads to more systematic generalization and
better convergence between humans and neural network agents and between
different neural agents. We then replicate this structure bias found in humans
and our recurrent neural networks with a Transformer-based large language model
(GPT-3), showing a similar benefit for structured linguistic input regarding
generalization systematicity and memorization errors. These findings show that
the underlying structure of languages is crucial for systematic generalization.
Due to the correlation between community size and linguistic structure in
natural languages, our findings underscore the challenge of automated
processing of low-resource languages. Nevertheless, the similarity between
humans and machines opens new avenues for language evolution research.

通过在人工语言学习研究中复制并测试深度神经网络学习新语言的记忆和推广能力，我们在神经网络与人类之间发现了惊人的相似性，并发现结构化语言输入在提高神经网络系统化概括和记忆错误方面的效果与自然语言存在高度相关性。

语言的深度学习易度是由什么决定的？

What makes a language easy to deep-learn?

Prominent questions about the role of sensory vs. linguistic input in the way
we acquire and use language have been extensively studied in the
psycholinguistic literature. However, the relative effect of various factors in
a person's overall experience on their linguistic system remains unclear. We
study this question by making a step forward towards a better understanding of
the conceptual perception of colors by color-blind individuals, as reflected in
their spontaneous linguistic productions. Using a novel and carefully curated
dataset, we show that red-green color-blind speakers use the "red" and "green"
color terms in less predictable contexts, and in linguistic environments
evoking mental image to a lower extent, when compared to their normal-sighted
counterparts. These findings shed some new and interesting light on the role of
sensory experience on our linguistic system.

研究了色盲患者对红绿色观念知觉的影响，发现他们的语言产出中红色和绿色词汇使用的可预测性较低且不常出现在心理图像引发的语言环境中，揭示了感官经验对于语言系统的影响。