Video moment retrieval and highlight detection are two highly valuable tasks
in video understanding, but until recently they have been jointly studied.
Although existing studies have made impressive advancement recently, they
predominantly follow the data-driven bottom-up paradigm. Such paradigm
overlooks task-specific and inter-task effects, resulting in poor model
performance. In this paper, we propose a novel task-driven top-down framework
TaskWeave for joint moment retrieval and highlight detection. The framework
introduces a task-decoupled unit to capture task-specific and common
representations. To investigate the interplay between the two tasks, we propose
an inter-task feedback mechanism, which transforms the results of one task as
guiding masks to assist the other task. Different from existing methods, we
present a task-dependent joint loss function to optimize the model.
Comprehensive experiments and in-depth ablation studies on QVHighlights, TVSum,
and Charades-STA datasets corroborate the effectiveness and flexibility of the
proposed framework. Codes are available at
this https URL

提出了一个基于任务驱动的自上而下框架 TaskWeave，用于联合视频片段检索和精彩时刻检测，通过任务解耦单元、任务特定共享表示以及任务之间的反馈机制实现了任务间的协同。实验验证了该框架的有效性和灵活性。