Learning to model and reconstruct humans in clothing is challenging due to
articulation, non-rigid deformation, and varying clothing types and topologies.
To enable learning, the choice of representation is the key. Recent work uses
neural networks to parameterize local surface elements. This approach captures
locally coherent geometry and non-planar details, can deal with varying
topology, and does not require registered training data. However, naively using
such methods to model 3D clothed humans fails to capture fine-grained local
deformations and generalizes poorly. To address this, we present three key
innovations: First, we deform surface elements based on a human body model such
that large-scale deformations caused by articulation are explicitly separated
from topological changes and local clothing deformations. Second, we address
the limitations of existing neural surface elements by regressing local
geometry from local features, significantly improving the expressiveness.
Third, we learn a pose embedding on a 2D parameterization space that encodes
posed body geometry, improving generalization to unseen poses by reducing
non-local spurious correlations. We demonstrate the efficacy of our surface
representation by learning models of complex clothing from point clouds. The
clothing can change topology and deviate from the topology of the body. Once
learned, we can animate previously unseen motions, producing high-quality point
clouds, from which we generate realistic images with neural rendering. We
assess the importance of each technical contribution and show that our approach
outperforms the state-of-the-art methods in terms of reconstruction accuracy
and inference time. The code is available for research purposes at
this https URL .

通过使用神经网络的表征来捕捉局部相干几何信息和非平面细节，并回归本地几何，我们提出了一种新的表面能量，通过在服装上基于人体模型的变形将人体和衣服的形变分离开来，在二维参数空间上学习一种姿态嵌入，对未知的姿态进行模型建立，以及通过动作瞬间学习人的姿态，实现了点云三维建模和渲染。

SCALE: 基于关节局部元素表面编解码的服装人体建模

SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local  Elements

The best performing methods for 3D human pose estimation from monocular
images require large amounts of in-the-wild 2D and controlled 3D pose annotated
datasets which are costly and require sophisticated systems to acquire. To
reduce this annotation dependency, we propose Multiview-Consistent Semi
Supervised Learning (MCSS) framework that utilizes similarity in pose
information from unannotated, uncalibrated but synchronized multi-view videos
of human motions as additional weak supervision signal to guide 3D human pose
regression. Our framework applies hard-negative mining based on temporal
relations in multi-view videos to arrive at a multi-view consistent pose
embedding. When jointly trained with limited 3D pose annotations, our approach
improves the baseline by 25% and state-of-the-art by 8.7%, whilst using
substantially smaller networks. Lastly, but importantly, we demonstrate the
advantages of the learned embedding and establish view-invariant pose retrieval
benchmarks on two popular, publicly available multi-view human pose datasets,
Human 3.6M and MPI-INF-3DHP, to facilitate future research.

该论文提出了一种基于多视角视频的多视角一致半监督学习框架，利用未经注释、未校准但同步的多视角视频中的姿态信息相似性作为额外的弱监督信号来引导 3D 人体姿势回归，该方法通过硬负采样来建立多视角一致的姿态嵌入，并结合有限的 3D 姿态注释来完善该模型，从而在两种公开数据集上实现了视角不变的姿态检索。

3D 人体姿势估计的多视图一致半监督学习

Multiview-Consistent Semi-Supervised Learning for 3D Human Pose  Estimation

In this work, we present an approach to deep visuomotor control using
structured deep dynamics models. Our deep dynamics model, a variant of
SE3-Nets, learns a low-dimensional pose embedding for visuomotor control via an
encoder-decoder structure. Unlike prior work, our dynamics model is structured:
given an input scene, our network explicitly learns to segment salient parts
and predict their pose-embedding along with their motion modeled as a change in
the pose space due to the applied actions. We train our model using a pair of
point clouds separated by an action and show that given supervision only in the
form of point-wise data associations between the frames our network is able to
learn a meaningful segmentation of the scene along with consistent poses. We
further show that our model can be used for closed-loop control directly in the
learned low-dimensional pose space, where the actions are computed by
minimizing error in the pose space using gradient-based methods, similar to
traditional model-based control. We present results on controlling a Baxter
robot from raw depth data in simulation and in the real world and compare
against two baseline deep networks. Our method runs in real-time, achieves good
prediction of scene dynamics and outperforms the baseline methods on multiple
control runs. Video results can be found at:
this https URL

本文介绍一种使用结构化的深度动态模型的方法实现深度视觉运动控制，使用一个低维度姿态嵌入来学习姿态嵌入为控制目标，使用点云数据实现监督训练，该模型能够实现实时、高效地控制机器人动作。