We address the problem of fitting a parametric human body model (SMPL) to
point cloud data. Optimization-based methods require careful initialization and
are prone to becoming trapped in local optima. Learning-based methods address
this but do not generalize well when the input pose is far from those seen
during training. For rigid point clouds, remarkable generalization has been
achieved by leveraging SE(3)-equivariant networks, but these methods do not
work on articulated objects. In this work we extend this idea to human bodies
and propose ArtEq, a novel part-based SE(3)-equivariant neural architecture for
SMPL model estimation from point clouds. Specifically, we learn a part
detection network by leveraging local SO(3) invariance, and regress shape and
pose using articulated SE(3) shape-invariant and pose-equivariant networks, all
trained end-to-end. Our novel equivariant pose regression module leverages the
permutation-equivariant property of self-attention layers to preserve
rotational equivariance. Experimental results show that ArtEq can generalize to
poses not seen during training, outperforming state-of-the-art methods by
74.5%, without requiring an optimization refinement step. Further, compared
with competing works, our method is more than three orders of magnitude faster
during inference and has 97.3% fewer parameters. The code and model will be
available for research purposes at this https URL

本文介绍了一种将参数化人体模型适应于点云数据的新方法 ——ArtEq。通过使用基于部分的 SE（3）- 等变神经结构来完成 SMPL 模型对点云的估计，并通过对等变姿态回归模块进行改进来实现对训练过程之外的姿态的泛化能力的提升。在实验中，ArtEq 方法的表现比同类方法提高了 74.5%，并且速度快了三个数量级。

利用关节可见性进行神经人体拟合的通用性及对未见姿态的推广

Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3)  Equivariance

Learning a generalizable object manipulation policy is vital for an embodied
agent to work in complex real-world scenes. Parts, as the shared components in
different object categories, have the potential to increase the generalization
ability of the manipulation policy and achieve cross-category object
manipulation. In this work, we build the first large-scale, part-based
cross-category object manipulation benchmark, PartManip, which is composed of
11 object categories, 494 objects, and 1432 tasks in 6 task classes. Compared
to previous work, our benchmark is also more diverse and realistic, i.e.,
having more objects and using sparse-view point cloud as input without oracle
information like part segmentation. To tackle the difficulties of vision-based
policy learning, we first train a state-based expert with our proposed
part-based canonicalization and part-aware rewards, and then distill the
knowledge to a vision-based student. We also find an expressive backbone is
essential to overcome the large diversity of different objects. For
cross-category generalization, we introduce domain adversarial learning for
domain-invariant feature extraction. Extensive experiments in simulation show
that our learned policy can outperform other methods by a large margin,
especially on unseen object categories. We also demonstrate our method can
successfully manipulate novel objects in the real world.

本研究提出了一种基于部件的交叉类别物体操纵策略，并使用新建的基准测试集 PartManip 进行了评估。通过标准化部件及关联奖励来培训状态专家，再通过知识蒸馏的方式将其应用于视觉学习中。此外，引入了领域敌对学习，以实现域不变性特征提取，结果表明该方法能够在模拟和真实环境中成功地操纵新颖物体。