Temporal action detection (TAD) involves the localization and classification
of action instances within untrimmed videos. While standard TAD follows fully
supervised learning with closed-set setting on large training data, recent
zero-shot TAD methods showcase the promising of open-set setting by leveraging
large-scale contrastive visual-language (ViL) pretrained models. However,
existing zero-shot TAD methods have limitations on how to properly construct
the strong relationships between two interdependent tasks of localization and
classification and adapt ViL model to video understanding. In this work, we
present ZEETAD, featuring two modules: dual-localization and zero-shot proposal
classification. The former is a Transformer-based module that detects action
events while selectively collecting crucial semantic embeddings for later
recognition. The latter one, CLIP-based module, generates semantic embeddings
from text and frame inputs for each temporal unit. Additionally, we enhance
discriminative capability on unseen classes by minimally updating the frozen
CLIP encoder with lightweight adapters. Extensive experiments on THUMOS14 and
ActivityNet-1.3 datasets demonstrate our approach's superior performance in
zero-shot TAD and effective knowledge transfer from ViL models to unseen action
categories.

本研究呈现了 ZEETAD，这是一个基于 Transformer 和 CLIP 模块的零样本时间动作检测方法，通过最小程度地更新冻结的 CLIP 编码器，实现对未见过的动作类别的辨别能力增强，并展示了其在零样本时间动作检测和从 ViL 模型到未见动作类别的有效知识传递方面的卓越性能。