The growing number of action classes has posed a new challenge for video
understanding, making Zero-Shot Action Recognition (ZSAR) a thriving direction.
The ZSAR task aims to recognize target (unseen) actions without training
examples by leveraging semantic representations to bridge seen and unseen
actions. However, due to the complexity and diversity of actions, it remains
challenging to semantically represent action classes and transfer knowledge
from seen data. In this work, we propose an ER-enhanced ZSAR model inspired by
an effective human memory technique Elaborative Rehearsal (ER), which involves
elaborating a new concept and relating it to known concepts. Specifically, we
expand each action class as an Elaborative Description (ED) sentence, which is
more discriminative than a class name and less costly than manual-defined
attributes. Besides directly aligning class semantics with videos, we
incorporate objects from the video as Elaborative Concepts (EC) to improve
video semantics and generalization from seen actions to unseen actions. Our
ER-enhanced ZSAR model achieves state-of-the-art results on three existing
benchmarks. Moreover, we propose a new ZSAR evaluation protocol on the Kinetics
dataset to overcome limitations of current benchmarks and demonstrate the first
case where ZSAR performance is comparable to few-shot learning baselines on
this more realistic setting. We will release our codes and collected EDs at
this https URL

本文提出了一种基于 Elaborative Rehearsal 技术的 ER-enhanced ZSAR 模型，该模型使用 Elaborative Description 和 Elaborative Concepts 来扩展动作类别以及提高视频语义，从而实现了零样本行为识别任务的最新成果，并在与 few-shot learning 模型的比较中提供针对 Kinetics 数据集的全新 ZSAR 评估方案。