Large language models (LLMs) have shown nearly saturated performance on many
natural language processing (NLP) tasks. As a result, it is natural for people
to believe that LLMs have also mastered abilities such as time understanding
and reasoning. However, research on the temporal sensitivity of LLMs has been
insufficiently emphasized. To fill this gap, this paper constructs Multiple
Sensitive Factors Time QA (MenatQA), which encompasses three temporal factors
(scope factor, order factor, counterfactual factor) with total 2,853 samples
for evaluating the time comprehension and reasoning abilities of LLMs. This
paper tests current mainstream LLMs with different parameter sizes, ranging
from billions to hundreds of billions. The results show most LLMs fall behind
smaller temporal reasoning models with different degree on these factors. In
specific, LLMs show a significant vulnerability to temporal biases and depend
heavily on the temporal information provided in questions. Furthermore, this
paper undertakes a preliminary investigation into potential improvement
strategies by devising specific prompts and leveraging external tools. These
approaches serve as valuable baselines or references for future research
endeavors.

本论文通过构建 MenatQA 来评估大型语言模型（LLMs）在时间理解和推理能力方面的表现，并测试了不同参数大小的主流 LLMs。结果表明，大多数 LLMs 在处理不同程度的时间因素时不如更小的时间推理模型，并且对时间偏差的敏感度较高，且严重依赖于问题中提供的时间信息。此外，本文还探索了通过具体提示和外部工具来改进 LLMs 的潜在策略，为未来的研究提供了有价值的基准或参考。

MenatQA：用于测试大型语言模型的时间理解和推理能力的新数据集

MenatQA: A New Dataset for Testing the Temporal Comprehension and  Reasoning Abilities of Large Language Models

Due to the large memory footprint of untrimmed videos, current
state-of-the-art video localization methods operate atop precomputed video clip
features. These features are extracted from video encoders typically trained
for trimmed action classification tasks, making such features not necessarily
suitable for temporal localization. In this work, we propose a novel supervised
pretraining paradigm for clip features that not only trains to classify
activities but also considers background clips and global video information to
improve temporal sensitivity. Extensive experiments show that using features
trained with our novel pretraining strategy significantly improves the
performance of recent state-of-the-art methods on three tasks: Temporal Action
Localization, Action Proposal Generation, and Dense Video Captioning. We also
show that our pretraining approach is effective across three encoder
architectures and two pretraining datasets. We believe video feature encoding
is an important building block for localization algorithms, and extracting
temporally-sensitive features should be of paramount importance in building
more accurate models. The code and pretrained models are available on our
project website.

该研究提出了一种新的监督预训练范例，通过考虑背景剪辑和全局视频信息，不仅需要训练活动分类，而且还需要训练时序灵敏度，从而显着提高了最近最先进的方法在三个任务中的性能：时间动作本地化，行动建议生成和密集视频字幕。