We introduce ReXTime, a benchmark designed to rigorously test AI models'
ability to perform temporal reasoning within video events. Specifically,
ReXTime focuses on reasoning across time, i.e. human-like understanding when
the question and its corresponding answer occur in different video segments.
This form of reasoning, requiring advanced understanding of cause-and-effect
relationships across video segments, poses significant challenges to even the
frontier multimodal large language models. To facilitate this evaluation, we
develop an automated pipeline for generating temporal reasoning question-answer
pairs, significantly reducing the need for labor-intensive manual annotations.
Our benchmark includes 921 carefully vetted validation samples and 2,143 test
samples, each manually curated for accuracy and relevance. Evaluation results
show that while frontier large language models outperform academic models, they
still lag behind human performance by a significant 14.3% accuracy gap.
Additionally, our pipeline creates a training dataset of 9,695 machine
generated samples without manual effort, which empirical studies suggest can
enhance the across-time reasoning via fine-tuning.

ReXTime 是一个基准测试，旨在严格测试 AI 模型在视频事件中执行时间推理的能力。它专注于跨时间的推理，即当问题及其相应答案发生在不同的视频片段时的人类理解。我们介绍了一个自动化流程，用于生成时间推理问题 - 答案对，大大减少了对手工注释的需求。评估结果显示，虽然前沿的大型语言模型优于学术模型，但它们仍然落后于人类表现，存在 14.3％的显著准确性差距。此外，我们的流程还创建了一个培训数据集，包含 9,695 个机器生成的样本，无需人工工作，经验研究表明这可以通过微调提升跨时间推理能力。

ReXTime: 视频跨时间推理的基准套件

ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos

We study the problem of facial analysis in videos. We propose a novel weakly
supervised learning method that models the video event (expression, pain etc.)
as a sequence of automatically mined, discriminative sub-events (eg. onset and
offset phase for smile, brow lower and cheek raise for pain). The proposed
model is inspired by the recent works on Multiple Instance Learning and latent
SVM/HCRF- it extends such frameworks to model the ordinal or temporal aspect in
the videos, approximately. We obtain consistent improvements over relevant
competitive baselines on four challenging and publicly available video based
facial analysis datasets for prediction of expression, clinical pain and intent
in dyadic conversations. In combination with complimentary features, we report
state-of-the-art results on these datasets.

本文研究了视频中的面部分析问题，提出了一种新的弱监督学习方法，模拟视频事件（表情，疼痛等）作为自动挖掘的有区别子事件的序列，这个模型受到了多个实例学习和潜在 SVM/HCRF 框架的启发，用于模仿视频中的顺序或时间方面， 本文的贡献主要在于提出了这样的结构并在四个具有挑战性且公开可用的基于视频的面部分析数据集上实现了优异的表现。