The use of neural language models to model human behavior has met with mixed
success. While some work has found that the surprisal estimates from these
models can be used to predict a wide range of human neural and behavioral
responses, other work studying more complex syntactic phenomena has found that
these surprisal estimates generate incorrect behavioral predictions. This paper
explores the extent to which the misalignment between empirical and
model-predicted behavior can be minimized by training models on more
developmentally plausible data, such as in the BabyLM Challenge. We trained
teacher language models on the BabyLM "strict-small" dataset and used sentence
level surprisal estimates from these teacher models to create a curriculum. We
found tentative evidence that our curriculum made it easier for models to
acquire linguistic knowledge from the training data: on the subset of tasks in
the BabyLM challenge suite evaluating models' grammatical knowledge of English,
models first trained on the BabyLM data curriculum and then on a few randomly
ordered training epochs performed slightly better than models trained on
randomly ordered epochs alone. This improved linguistic knowledge acquisition
did not result in better alignment with human reading behavior, however: models
trained on the BabyLM dataset (with or without a curriculum) generated
predictions that were as misaligned with human behavior as models trained on
larger less curated datasets. This suggests that training on developmentally
plausible datasets alone is likely insufficient to generate language models
capable of accurately predicting human language processing.

使用神经语言模型对人类行为进行建模在研究中获得了不同的结果。本文通过在 BabyLM 挑战中使用更贴近发展的数据集来探索实证数据和模型预测行为之间的不匹配程度。通过对 BabyLM 的数据集进行师生训练和课程设计，研究发现，虽然这种改进使得模型更容易从训练数据中获取语言知识，但并未导致模型对人类阅读行为的预测与之更加一致，这表明仅仅在发展上合理的数据集上训练模型可能不足以准确预测人类语言处理。