In this work, we introduce a novel task - Humancentric Spatio-Temporal Video
Grounding (HC-STVG). Unlike the existing referring expression tasks in images
or videos, by focusing on humans, HC-STVG aims to localize a spatiotemporal
tube of the target person from an untrimmed video based on a given textural
description. This task is useful, especially for healthcare and
security-related applications, where the surveillance videos can be extremely
long but only a specific person during a specific period of time is concerned.
HC-STVG is a video grounding task that requires both spatial (where) and
temporal (when) localization. Unfortunately, the existing grounding methods
cannot handle this task well. We tackle this task by proposing an effective
baseline method named Spatio-Temporal Grounding with Visual Transformers
(STGVT), which utilizes Visual Transformers to extract cross-modal
representations for video-sentence matching and temporal localization. To
facilitate this task, we also contribute an HC-STVG dataset consisting of 5,660
video-sentence pairs on complex multi-person scenes. Specifically, each video
lasts for 20 seconds, pairing with a natural query sentence with an average of
17.25 words. Extensive experiments are conducted on this dataset, demonstrating
the newly-proposed method outperforms the existing baseline methods.

本文介绍了一项新任务 —— 人体中心的时空视频定位（HC-STVG），并提出了一种名为 Spatio-Temporal Grounding with Visual Transformers（STGVT）的基准方法，该方法使用 Visual Transformers 来提取视频 - 句子匹配和时间定位的跨模态表示，同时我们贡献了一个 HC-STVG 数据集，其中包含 5,660 个复杂的多人场景视频 - 句子对。经过大量实验，表明新提出的方法优于现有的基准方法。