In this paper we introduce a rule-based, compositional, and hierarchical
modeling of action using Therbligs as our atoms. Introducing these atoms
provides us with a consistent, expressive, contact-centered representation of
action. Over the atoms we introduce a differentiable method of rule-based
reasoning to regularize for logical consistency. Our approach is complementary
to other approaches in that the Therblig-based representations produced by our
architecture augment rather than replace existing architectures'
representations. We release the first Therblig-centered annotations over two
popular video datasets - EPIC Kitchens 100 and 50-Salads. We also broadly
demonstrate benefits to adopting Therblig representations through evaluation on
the following tasks: action segmentation, action anticipation, and action
recognition - observing an average 10.5\%/7.53\%/6.5\% relative improvement,
respectively, over EPIC Kitchens and an average 8.9\%/6.63\%/4.8\% relative
improvement, respectively, over 50 Salads. Code and data will be made publicly
available.

本篇论文介绍了使用 Therbligs 作为原子的基于规则、组合和分层的行动建模方法，通过可微的规则推理方法，我们实现了逻辑一致性的规则化。我们在两个流行的视频数据集上发布了第一个基于 Therblig 的注释，并通过对以下任务的评估（动作分割，动作预测和动作识别）广泛证明了采用 Therblig 表示法的好处，相对改善分别为 10.5％/ 7.53％/ 6.5％和 8.9％/ 6.63％/ 4.8％。

动作原理理解下的 Therbligs 应用

Therbligs in Action: Video Understanding through Motion Primitives

In this work we explore reconstructing hand-object interactions in the wild.
The core challenge of this problem is the lack of appropriate 3D labeled data.
To overcome this issue, we propose an optimization-based procedure which does
not require direct 3D supervision. The general strategy we adopt is to exploit
all available related data (2D bounding boxes, 2D hand keypoints, 2D instance
masks, 3D object models, 3D in-the-lab MoCap) to provide constraints for the 3D
reconstruction. Rather than optimizing the hand and object individually, we
optimize them jointly which allows us to impose additional constraints based on
hand-object contact, collision, and occlusion. Our method produces compelling
reconstructions on the challenging in-the-wild data from the EPIC Kitchens and
the 100 Days of Hands datasets, across a range of object categories.
Quantitatively, we demonstrate that our approach compares favorably to existing
approaches in the lab settings where ground truth 3D annotations are available.

本研究旨在探讨如何在自然环境中重建手 - 物互动。为此，我们提出了一个基于优化的过程，该过程不需要直接的 3D 监督，并利用所有可用的相关数据来提供 3D 重建的约束条件。我们的方法在 EPIC Kitchens 和 100 Days of Hands 数据集上跨越一系列对象类别产生了引人注目的重建效果，并在实验室的环境中与现有方法进行了比较。