Given a long untrimmed video and natural language queries, video grounding
(VG) aims to temporally localize the semantically-aligned video segments.
Almost all existing VG work holds two simple but unrealistic assumptions: 1)
All query sentences can be grounded in the corresponding video. 2) All query
sentences for the same video are always at the same semantic scale.
Unfortunately, both assumptions make today's VG models fail to work in
practice. For example, in real-world multimodal assets (eg, news articles),
most of the sentences in the article can not be grounded in their affiliated
videos, and they typically have rich hierarchical relations (ie, at different
semantic scales). To this end, we propose a new challenging grounding task:
Weakly-Supervised temporal Article Grounding (WSAG). Specifically, given an
article and a relevant video, WSAG aims to localize all ``groundable''
sentences to the video, and these sentences are possibly at different semantic
scales. Accordingly, we collect the first WSAG dataset to facilitate this task:
YouwikiHow, which borrows the inherent multi-scale descriptions in wikiHow
articles and plentiful YouTube videos. In addition, we propose a simple but
effective method DualMIL for WSAG, which consists of a two-level MIL loss and a
single-/cross- sentence constraint loss. These training objectives are
carefully designed for these relaxed assumptions. Extensive ablations have
verified the effectiveness of DualMIL.

通过提出一个新的挑战性任务 Weakly-Supervised temporal Article Grounding (WSAG) 和一个有效的方法 DualMIL，可以在一个相关的多媒体资源 (文章和视频) 的不同层级之间定位相关句子。我们提出了第一个 WSAG 数据集 YouwikHow，利用 wikiHow 文章和 YouTube 视频中的多尺度描述，并证明了 DualMIL 的有效性。