By positing a relationship between naturalistic reading times and
information-theoretic surprisal, surprisal theory (Hale, 2001; Levy, 2008)
provides a natural interface between language models and psycholinguistic
models. This paper re-evaluates a claim due to Goodkind and Bicknell (2018)
that a language model's ability to model reading times is a linear function of
its perplexity. By extending Goodkind and Bicknell's analysis to modern neural
architectures, we show that the proposed relation does not always hold for Long
Short-Term Memory networks, Transformers, and pre-trained models. We introduce
an alternate measure of language modeling performance called predictability
norm correlation based on Cloze probabilities measured from human subjects. Our
new metric yields a more robust relationship between language model quality and
psycholinguistic modeling performance that allows for comparison between models
with different training configurations.

通过对现代神经结构的分析，提出一种新的语言建模表现度量并与人类主观认知语言处理结果的相关性来重新评估 Goodkind 和 Bicknell（2018 年）的观点，证明了一种基于困惑度的语言模型能否对阅读时间进行建模的线性假设不适用于 LSTM 网络，变形器和预训练模型。