In this work, we present a novel approach to multi-view action recognition
where we guide learned action representations to be separated from
view-relevant information in a video. When trying to classify action instances
captured from multiple viewpoints, there is a higher degree of difficulty due
to the difference in background, occlusion, and visibility of the captured
action from different camera angles. To tackle the various problems introduced
in multi-view action recognition, we propose a novel configuration of learnable
transformer decoder queries, in conjunction with two supervised contrastive
losses, to enforce the learning of action features that are robust to shifts in
viewpoints. Our disentangled feature learning occurs in two stages: the
transformer decoder uses separate queries to separately learn action and view
information, which are then further disentangled using our two contrastive
losses. We show that our model and method of training significantly outperforms
all other uni-modal models on four multi-view action recognition datasets: NTU
RGB+D, NTU RGB+D 120, PKU-MMD, and N-UCLA. Compared to previous RGB works, we
see maximal improvements of 1.5\%, 4.8\%, 2.2\%, and 4.8\% on each dataset,
respectively.

提出一种新颖的多视图动作识别方法，通过可学习的变换器解码查询和两个监督对比损失将学习到的动作特征与视图信息分离，从而显著提高了多视图动作识别的性能。