The majority of research in computational psycholinguistics has concentrated
on the processing of words. This study introduces innovative methods for
computing sentence-level metrics using multilingual large language models. The
metrics developed sentence surprisal and sentence relevance and then are tested
and compared to validate whether they can predict how humans comprehend
sentences as a whole across languages. These metrics offer significant
interpretability and achieve high accuracy in predicting human sentence reading
speeds. Our results indicate that these computational sentence-level metrics
are exceptionally effective at predicting and elucidating the processing
difficulties encountered by readers in comprehending sentences as a whole
across a variety of languages. Their impressive performance and generalization
capabilities provide a promising avenue for future research in integrating LLMs
and cognitive science.

创新的方法使用多语言大型语言模型计算句子级别的度量标准，预测并阐明读者在理解句子时遇到的处理困难，提供了实质性的可解释性和高准确性，为将大型语言模型和认知科学整合的未来研究提供了有希望的途径。

计算句级度量预测人类句子理解

Computational Sentence-level Metrics Predicting Human Sentence  Comprehension

In computational psycholinguistics, various language models have been
evaluated against human reading behavior (e.g., eye movement) to build
human-like computational models. However, most previous efforts have focused
almost exclusively on English, despite the recent trend towards linguistic
universal within the general community. In order to fill the gap, this paper
investigates whether the established results in computational psycholinguistics
can be generalized across languages. Specifically, we re-examine an established
generalization -- the lower perplexity a language model has, the more
human-like the language model is -- in Japanese with typologically different
structures from English. Our experiments demonstrate that this established
generalization exhibits a surprising lack of universality; namely, lower
perplexity is not always human-like. Moreover, this discrepancy between English
and Japanese is further explored from the perspective of (non-)uniform
information density. Overall, our results suggest that a cross-lingual
evaluation will be necessary to construct human-like computational models.

本文旨在研究跨语言时，计算语言学中已有的总结是否可以被推广以及在确定信息密度统一性的前提下，较低的困惑度并不总是人类化语言模型的表现。结果表明，为了构建类似于人类的计算模型，需要进行跨语言评估。