Skeletal Action recognition from an egocentric view is important for
applications such as interfaces in AR/VR glasses and human-robot interaction,
where the device has limited resources. Most of the existing skeletal action
recognition approaches use 3D coordinates of hand joints and 8-corner
rectangular bounding boxes of objects as inputs, but they do not capture how
the hands and objects interact with each other within the spatial context. In
this paper, we present a new framework called Contact-aware Skeletal Action
Recognition (CaSAR). It uses novel representations of hand-object interaction
that encompass spatial information: 1) contact points where the hand joints
meet the objects, 2) distant points where the hand joints are far away from the
object and nearly not involved in the current action. Our framework is able to
learn how the hands touch or stay away from the objects for each frame of the
action sequence, and use this information to predict the action class. We
demonstrate that our approach achieves the state-of-the-art accuracy of 91.3%
and 98.4% on two public datasets, H2O and FPHA, respectively.

以供应资源有限的 AR/VR 眼镜界面和人机交互等应用为背景下，从第一人称视角进行骨架动作识别具有重要意义。本篇研究论文引入了一种称为 CaSAR（接触感知骨架动作识别）的新框架，它采用了手 - 物体交互的新表示形式来捕捉空间信息，并通过学习手与物体的接触与分离情况来预测动作类别，其中实现了在 H2O 和 FPHA 两个公共数据集上的 91.3% 和 98.4% 的最新准确性。

CaSAR：接触感知骨骼动作识别

CaSAR: Contact-aware Skeletal Action Recognition

Many skeletal action recognition models use GCNs to represent the human body
by 3D body joints connected body parts. GCNs aggregate one- or few-hop graph
neighbourhoods, and ignore the dependency between not linked body joints. We
propose to form hypergraph to model hyper-edges between graph nodes (e.g.,
third- and fourth-order hyper-edges capture three and four nodes) which help
capture higher-order motion patterns of groups of body joints. We split action
sequences into temporal blocks, Higher-order Transformer (HoT) produces
embeddings of each temporal block based on (i) the body joints, (ii) pairwise
links of body joints and (iii) higher-order hyper-edges of skeleton body
joints. We combine such HoT embeddings of hyper-edges of orders 1, ..., r by a
novel Multi-order Multi-mode Transformer (3Mformer) with two modules whose
order can be exchanged to achieve coupled-mode attention on coupled-mode tokens
based on 'channel-temporal block', 'order-channel-body joint',
'channel-hyper-edge (any order)' and 'channel-only' pairs. The first module,
called Multi-order Pooling (MP), additionally learns weighted aggregation along
the hyper-edge mode, whereas the second module, Temporal block Pooling (TP),
aggregates along the temporal block mode. Our end-to-end trainable network
yields state-of-the-art results compared to GCN-, transformer- and
hypergraph-based counterparts.

该研究提出了一种使用超图模型的多阶多模态转换器来识别骨骼动作，并包括更高级别的运动模式及可训练的网络，在骨骼活动识别中优于 GCN、transformer 和超图等现有模型。

3Mformer：多阶多模态 Transformer 用于骨骼动作识别

3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition

Human actions comprise of joint motion of articulated body parts or
`gestures'. Human skeleton is intuitively represented as a sparse graph with
joints as nodes and natural connections between them as edges. Graph
convolutional networks have been used to recognize actions from skeletal
videos. We introduce a part-based graph convolutional network (PB-GCN) for this
task, inspired by Deformable Part-based Models (DPMs). We divide the skeleton
graph into four subgraphs with joints shared across them and learn a
recognition model using a part-based graph convolutional network. We show that
such a model improves performance of recognition, compared to a model using
entire skeleton graph. Instead of using 3D joint coordinates as node features,
we show that using relative coordinates and temporal displacements boosts
performance. Our model achieves state-of-the-art performance on two challenging
benchmark datasets NTURGB+D and HDM05, for skeletal action recognition.

本文提出了一种基于部件图卷积网络的人体动作识别模型，该模型将骨骼图分为四个子图，并使用相对坐标和时间位移代替 3D 关节点特征，相对于使用整个骨骼图的模型，能够提高识别性能，实现了对挑战性基准数据集 NTURGB+D 和 HDM05 的骨骼动作识别的最先进水平。