While most modern video understanding models operate on short-range clips,
real-world videos are often several minutes long with semantically consistent
segments of variable length. A common approach to process long videos is
applying a short-form video model over uniformly sampled clips of fixed
temporal length and aggregating the outputs. This approach neglects the
underlying nature of long videos since fixed-length clips are often redundant
or uninformative. In this paper, we aim to provide a generic and adaptive
sampling approach for long-form videos in lieu of the de facto uniform
sampling. Viewing videos as semantically consistent segments, we formulate a
task-agnostic, unsupervised, and scalable approach based on Kernel Temporal
Segmentation (KTS) for sampling and tokenizing long videos. We evaluate our
method on long-form video understanding tasks such as video classification and
temporal action localization, showing consistent gains over existing approaches
and achieving state-of-the-art performance on long-form video modeling.

本文旨在提供一种针对长视频的通用和自适应采样方法，通过将视频视为语义一致的片段，基于核时序分割 (KTS) 的无监督和可扩展方法对长视频进行采样和标记化。我们在视频分类和时间动作定位等长视频理解任务上评估了我们的方法，显示出相对于现有方法的持续增益，并在长视频建模上取得了最先进的性能。