Zero-shot human skeleton-based action recognition aims to construct a model
that can recognize actions outside the categories seen during training.
Previous research has focused on aligning sequences' visual and semantic
spatial distributions. However, these methods extract semantic features simply.
They ignore that proper prompt design for rich and fine-grained action cues can
provide robust representation space clustering. In order to alleviate the
problem of insufficient information available for skeleton sequences, we design
an information compensation learning framework from an information-theoretic
perspective to improve zero-shot action recognition accuracy with a
multi-granularity semantic interaction mechanism. Inspired by ensemble
learning, we propose a multi-level alignment (MLA) approach to compensate
information for action classes. MLA aligns multi-granularity embeddings with
visual embedding through a multi-head scoring mechanism to distinguish
semantically similar action names and visually similar actions. Furthermore, we
introduce a new loss function sampling method to obtain a tight and robust
representation. Finally, these multi-granularity semantic embeddings are
synthesized to form a proper decision surface for classification. Significant
action recognition performance is achieved when evaluated on the challenging
NTU RGB+D, NTU RGB+D 120, and PKU-MMD benchmarks and validate that
multi-granularity semantic features facilitate the differentiation of action
clusters with similar visual features.

从信息论的角度设计信息补偿学习框架，通过多粒度语义交互机制来提高零样本动作识别准确性，并提出多层次对齐方法对动作类别的信息进行补偿，利用新的损失函数采样方法获得紧密且鲁棒的表示，进而通过合成多粒度语义嵌入形成适当的分类决策面，验证多粒度语义特征有助于区分具有相似视觉特征的动作簇。