Occlusions are universal disruptions constantly present in the real world.
Especially for sparse representations, such as human skeletons, a few occluded
points might destroy the geometrical and temporal continuity critically
affecting the results. Yet, the research of data-scarce recognition from
skeleton sequences, such as one-shot action recognition, does not explicitly
consider occlusions despite their everyday pervasiveness. In this work, we
explicitly tackle body occlusions for Skeleton-based One-shot Action
Recognition (SOAR). We mainly consider two occlusion variants: 1) random
occlusions and 2) more realistic occlusions caused by diverse everyday objects,
which we generate by projecting the existing IKEA 3D furniture models into the
camera coordinate system of the 3D skeletons with different geometric
parameters. We leverage the proposed pipeline to blend out portions of skeleton
sequences of the three popular action recognition datasets and formalize the
first benchmark for SOAR from partially occluded body poses. Another key
property of our benchmark are the more realistic occlusions generated by
everyday objects, as even in standard recognition from 3D skeletons, only
randomly missing joints were considered. We re-evaluate existing
state-of-the-art frameworks for SOAR in the light of this new task and further
introduce Trans4SOAR - a new transformer-based model which leverages three data
streams and mixed attention fusion mechanism to alleviate the adverse effects
caused by occlusions. While our experiments demonstrate a clear decline in
accuracy with missing skeleton portions, this effect is smaller with
Trans4SOAR, which outperforms other architectures on all datasets. Although we
specifically focus on occlusions, Trans4SOAR additionally yields
state-of-the-art in the standard SOAR without occlusion, surpassing the best
published approach by 2.85% on NTU-120.

本研究为纠正骨架动作识别中身体遮挡问题提出了 Trans4SOAR 模型，该模型利用三个数据流及混合注意力融合机制缓解遮挡造成的副作用，并在标准 SOAR 和部分遮挡 SOAR 上均取得了最新成果。