In real-world scenarios, human actions often fall outside the distribution of
training data, making it crucial for models to recognize known actions and
reject unknown ones. However, using pure skeleton data in such open-set
conditions poses challenges due to the lack of visual background cues and the
distinct sparse structure of body pose sequences. In this paper, we tackle the
unexplored Open-Set Skeleton-based Action Recognition (OS-SAR) task and
formalize the benchmark on three skeleton-based datasets. We assess the
performance of seven established open-set approaches on our task and identify
their limits and critical generalization issues when dealing with skeleton
information. To address these challenges, we propose a distance-based
cross-modality ensemble method that leverages the cross-modal alignment of
skeleton joints, bones, and velocities to achieve superior open-set recognition
performance. We refer to the key idea as CrossMax - an approach that utilizes a
novel cross-modality mean max discrepancy suppression mechanism to align latent
spaces during training and a cross-modality distance-based logits refinement
method during testing. CrossMax outperforms existing approaches and
consistently yields state-of-the-art results across all datasets and backbones.
The benchmark, code, and models will be released at
this https URL

本文探讨了开放集骨架动作识别（OS-SAR）任务的挑战，并提出了一种基于距离的跨模态集成方法，通过对骨骼关节、骨头和速度进行交叉模态对齐，在训练过程中使用交叉模态均值最大差异抑制机制，测试过程中使用基于距离的交叉模态逻辑改进方法，从而实现优秀的开放集识别表现。该方法在各个数据集和底层模型上持续取得最先进的结果。