Large language models (LLMs) have recently gained significant attention due
to their unparalleled ability to perform various natural language processing
tasks. These models, benefiting from their advanced natural language
understanding capabilities, have demonstrated impressive zero-shot performance.
However, the pre-training data utilized in LLMs is often confined to a specific
corpus, resulting in inherent freshness and temporal scope limitations.
Consequently, this raises concerns regarding the effectiveness of LLMs for
tasks involving temporal intents. In this study, we aim to investigate the
underlying limitations of general-purpose LLMs when deployed for tasks that
require a temporal understanding. We pay particular attention to handling
factual temporal knowledge through three popular temporal QA datasets.
Specifically, we observe low performance on detailed questions about the past
and, surprisingly, for rather new information. In manual and automatic testing,
we find multiple temporal errors and characterize the conditions under which QA
performance deteriorates. Our analysis contributes to understanding LLM
limitations and offers valuable insights into developing future models that can
better cater to the demands of temporally-oriented tasks. The code is
available\footnote{this https URL}.

该研究旨在调查通用型大型语言模型在需要时间理解的任务中面临的潜在限制，针对三个流行的时间问答数据集，我们发现大型语言模型在关于过去和新信息的详细问题上表现较差，存在多个时间错误，我们的分析有助于理解大型语言模型的局限性，并为开发更好满足时间导向任务需求的未来模型提供有价值的见解。