Video Foundation Models (ViFMs) aim to learn a general-purpose representation
for various video understanding tasks. Leveraging large-scale datasets and
powerful models, ViFMs achieve this by capturing robust and generic features
from video data. This survey analyzes over 200 video foundational models,
offering a comprehensive overview of benchmarks and evaluation metrics across
14 distinct video tasks categorized into 3 main categories. Additionally, we
offer an in-depth performance analysis of these models for the 6 most common
video tasks. We categorize ViFMs into three categories: 1) Image-based ViFMs,
which adapt existing image models for video tasks, 2) Video-Based ViFMs, which
utilize video-specific encoding methods, and 3) Universal Foundational Models
(UFMs), which combine multiple modalities (image, video, audio, and text etc.)
within a single framework. By comparing the performance of various ViFMs on
different tasks, this survey offers valuable insights into their strengths and
weaknesses, guiding future advancements in video understanding. Our analysis
surprisingly reveals that image-based foundation models consistently outperform
video-based models on most video understanding tasks. Additionally, UFMs, which
leverage diverse modalities, demonstrate superior performance on video tasks.
We share the comprehensive list of ViFMs studied in this work at:
https://github.com/NeeluMadan/ViFM_Survey.git

此研究简化了近 200 种视频基础模型，对 14 个不同的视频任务进行了综合概述，并在这些任务中对 6 个最常见的任务进行了性能分析。该研究发现，图像为基础的模型在大多数视频理解任务上持续表现优异，而利用多种模式的通用基础模型在视频任务上表现卓越。