To integrate action recognition methods into autonomous robotic systems, it
is crucial to consider adverse situations involving target occlusions. Such a
scenario, despite its practical relevance, is rarely addressed in existing
self-supervised skeleton-based action recognition methods. To empower robots
with the capacity to address occlusion, we propose a simple and effective
method. We first pre-train using occluded skeleton sequences, then use k-means
clustering (KMeans) on sequence embeddings to group semantically similar
samples. Next, we employ K-nearest-neighbor (KNN) to fill in missing skeleton
data based on the closest sample neighbors. Imputing incomplete skeleton
sequences to create relatively complete sequences as input provides significant
benefits to existing skeleton-based self-supervised models. Meanwhile, building
on the state-of-the-art Partial Spatio-Temporal Learning (PSTL), we introduce
an Occluded Partial Spatio-Temporal Learning (OPSTL) framework. This
enhancement utilizes Adaptive Spatial Masking (ASM) for better use of
high-quality, intact skeletons. The effectiveness of our imputation methods is
verified on the challenging occluded versions of the NTURGB+D 60 and NTURGB+D
120. The source code will be made publicly available at
this https URL

将动作识别方法集成到自主机器人系统中，考虑目标遮挡的不利情况是至关重要的。我们提出了一种简单有效的方法，通过预训练使用遮挡的骨架序列，然后使用 KMeans 对序列嵌入进行聚类，以填补缺失的骨架数据，从而为现有基于骨架的自监督模型带来显著的好处。同时，在 Partial Spatio-Temporal Learning (PSTL) 的基础上，我们引入了一种新的 Occluded Partial Spatio-Temporal Learning (OPSTL) 框架，利用 Adaptive Spatial Masking (ASM) 更好地利用高质量完整的骨架数据。我们的填补方法的有效性在具有挑战性的 NTURGB+D 60 和 NTURGB+D 120 的遮挡版本上得到了验证。