Large Language Models (LLMs) are increasingly becoming ubiquitous, yet their
ability to reason about and retain temporal information remains limited. This
hinders their application in real-world scenarios where understanding the
sequential nature of events is crucial. This paper experiments with
state-of-the-art models on a novel, large-scale temporal dataset,
\textbf{TempUN}, to reveal significant limitations in temporal retention and
reasoning abilities. Interestingly, closed-source models indicate knowledge
gaps more frequently, potentially suggesting a trade-off between uncertainty
awareness and incorrect responses. Further, exploring various fine-tuning
approaches yielded no major performance improvements. The associated dataset
and code are available at the following URL
(this https URL).

大型语言模型对于时间信息的推理和保留能力仍然有限，本论文通过在一个大规模时间数据集（TempUN）上实验，揭示了在时间保留和推理能力方面存在的显著局限。有趣的是，闭源模型更频繁地显示出知识缺口，可能表明在不确定性意识和错误响应之间存在折衷。此外，探索各种微调方法并没有带来显著的性能提升。

评估大型语言模型中的时间信息和推理技能

Remember This Event That Year? Assessing Temporal Information and  Reasoning in Large Language Models

Understanding temporal information and how the visual world changes over time
is a fundamental ability of intelligent systems. In video understanding,
temporal information is at the core of many current challenges, including
compression, efficient inference, motion estimation or summarization. However,
in current video datasets it has been observed that action classes can often be
recognized without any temporal information from a single frame of video. As a
result, both benchmarking and training in these datasets may give an
unintentional advantage to models with strong image understanding capabilities,
as opposed to those with strong temporal understanding. In this paper we
address this problem head on by identifying action classes where temporal
information is actually necessary to recognize them and call these "temporal
classes". Selecting temporal classes using a computational method would bias
the process. Instead, we propose a methodology based on a simple and effective
human annotation experiment. We remove just the temporal information by
shuffling frames in time and measure if the action can still be recognized.
Classes that cannot be recognized when frames are not in order are included in
the temporal Dataset. We observe that this set is statistically different from
other static classes, and that performance in it correlates with a network's
ability to capture temporal information. Thus we use it as a benchmark on
current popular networks, which reveals a series of interesting facts. We also
explore the effect of training on the temporal dataset, and observe that this
leads to better generalization in unseen classes, demonstrating the need for
more temporal data. We hope that the proposed dataset of temporal categories
will help guide future research in temporal modeling for better video
understanding.

该研究通过构建一个基于 ' 时间数据集 ' 的分类器来研究理解视觉世界如何随时间变化以及在视频理解中，Temporal 与 Static Action Classes 的区别；并表明更多的时间数据可以帮助提高模型性能。