Action recognition is essential for egocentric video understanding, allowing
automatic and continuous monitoring of Activities of Daily Living (ADLs)
without user effort. Existing literature focuses on 3D hand pose input, which
requires computationally intensive depth estimation networks or wearing an
uncomfortable depth sensor. In contrast, there has been insufficient research
in understanding 2D hand pose for egocentric action recognition, despite the
availability of user-friendly smart glasses in the market capable of capturing
a single RGB image. Our study aims to fill this research gap by exploring the
field of 2D hand pose estimation for egocentric action recognition, making two
contributions. Firstly, we introduce two novel approaches for 2D hand pose
estimation, namely EffHandNet for single-hand estimation and EffHandEgoNet,
tailored for an egocentric perspective, capturing interactions between hands
and objects. Both methods outperform state-of-the-art models on H2O and FPHA
public benchmarks. Secondly, we present a robust action recognition
architecture from 2D hand and object poses. This method incorporates
EffHandEgoNet, and a transformer-based action recognition method. Evaluated on
H2O and FPHA datasets, our architecture has a faster inference time and
achieves an accuracy of 91.32% and 94.43%, respectively, surpassing state of
the art, including 3D-based methods. Our work demonstrates that using 2D
skeletal data is a robust approach for egocentric action understanding.
Extensive evaluation and ablation studies show the impact of the hand pose
estimation approach, and how each input affects the overall performance.

通过探索 2D 手势姿态评估用于自我中心动作识别的领域，我们提出了两种新方法：EffHandNet 用于单手姿态估计和 EffHandEgoNet 用于自我视角，捕捉手部与物体之间的交互。同时，我们提出了一个从 2D 手部和物体姿态的坚固的动作识别架构。通过在 H2O 和 FPHA 数据集上的评估，我们的架构具有更快的推断时间，并且分别达到了 91.32% 和 94.43% 的精度，超越了包括基于 3D 的方法在内的最先进的技术。

我的视角，我的双手：准确的自视角二维手势姿态和动作识别

In My Perspective, In My Hands: Accurate Egocentric 2D Hand Pose and  Action Recognition

In this paper, we propose a new architecture called Adaptive Graphical Model
Network (AGMN) to tackle the task of 2D hand pose estimation from a monocular
RGB image. The AGMN consists of two branches of deep convolutional neural
networks for calculating unary and pairwise potential functions, followed by a
graphical model inference module for integrating unary and pairwise potentials.
Unlike existing architectures proposed to combine DCNNs with graphical models,
our AGMN is novel in that the parameters of its graphical model are conditioned
on and fully adaptive to individual input images. Experiments show that our
approach outperforms the state-of-the-art method used in 2D hand keypoints
estimation by a notable margin on two public datasets.

本文提出一种名为自适应图模型网络（AGMN）的新架构，用于从单个 RGB 图像估计 2D 手部姿势，该架构包含两个深度卷积神经网络分支，用于计算一元和双元潜力函数，随后是图形模型推断模块，用于集成一元和双元潜力。与现有的将 DCNN 与图形模型结合的体系结构不同，我们的 AGMN 是新颖的，因为其图形模型的参数是基于并完全适应于个别输入图像的。实验证明，我们的方法在两个公共数据集上比 2D 手关键点估计中使用的最先进方法优越了不少。

自适应图模型网络用于二维手部姿态估计

Adaptive Graphical Model Network for 2D Handpose Estimation

Humans excel at grasping objects and manipulating them. Capturing human
grasps is important for understanding grasping behavior and reconstructing it
realistically in Virtual Reality (VR). However, grasp capture - capturing the
pose of a hand grasping an object, and orienting it w.r.t. the object - is
difficult because of the complexity and diversity of the human hand, and
occlusion. Reflective markers and magnetic trackers traditionally used to
mitigate this difficulty introduce undesirable artifacts in images and can
interfere with natural grasping behavior. We present preliminary work on a
completely marker-less algorithm for grasp capture from a video depicting a
grasp. We show how recent advances in 2D hand pose estimation can be used with
well-established optimization techniques. Uniquely, our algorithm can also
capture hand-object contact in detail and integrate it in the grasp capture
process. This is work in progress, find more details at this https URL
cc.gatech.edu/grasp_capture.html.

研究了通过使用最新的二维手姿势估计技术和优化技术进行的无标记算法来捕捉手抓握物体的过程，此算法也可以详细地捕捉手 - 物体接触并将其集成进抓握捕捉过程中。