Creating animatable avatars from static scans requires the modeling of
clothing deformations in different poses. Existing learning-based methods
typically add pose-dependent deformations upon a minimally-clothed mesh
template or a learned implicit template, which have limitations in capturing
details or hinder end-to-end learning. In this paper, we revisit point-based
solutions and propose to decompose explicit garment-related templates and then
add pose-dependent wrinkles to them. In this way, the clothing deformations are
disentangled such that the pose-dependent wrinkles can be better learned and
applied to unseen poses. Additionally, to tackle the seam artifact issues in
recent state-of-the-art point-based methods, we propose to learn point features
on a body surface, which establishes a continuous and compact feature space to
capture the fine-grained and pose-dependent clothing geometry. To facilitate
the research in this field, we also introduce a high-quality scan dataset of
humans in real-world clothing. Our approach is validated on two existing
datasets and our newly introduced dataset, showing better clothing deformation
results in unseen poses. The project page with code and dataset can be found at
this https URL

该研究提出了一种基于点的解决方案来解决服装变形问题，并引入了一个包含现实世界人类穿着高质量扫描数据集的方法进行验证。

CloSET: 使用显式模板分解对穿着衣服的人类进行连续表面建模

CloSET: Modeling Clothed Humans on Continuous Surface with Explicit  Template Decomposition

We have recently seen great progress in building photorealistic animatable
full-body codec avatars, but generating high-fidelity animation of clothing is
still difficult. To address these difficulties, we propose a method to build an
animatable clothed body avatar with an explicit representation of the clothing
on the upper body from multi-view captured videos. We use a two-layer mesh
representation to register each 3D scan separately with the body and clothing
templates. In order to improve the photometric correspondence across different
frames, texture alignment is then performed through inverse rendering of the
clothing geometry and texture predicted by a variational autoencoder. We then
train a new two-layer codec avatar with separate modeling of the upper clothing
and the inner body layer. To learn the interaction between the body dynamics
and clothing states, we use a temporal convolution network to predict the
clothing latent code based on a sequence of input skeletal poses. We show
photorealistic animation output for three different actors, and demonstrate the
advantage of our clothed-body avatars over the single-layer avatars used in
previous work. We also show the benefit of an explicit clothing model that
allows the clothing texture to be edited in the animation output.

通过多视角捕捉视频对具有显式服装表示的可动衣物人形头像进行建模，采用二层网格表示注册每个 3D 扫描，通过时序卷积网络来预测衣物潜在编码以实现人体动力学和服装状态的交互作用。

为可动人形模型建模衣服的单独层

Modeling Clothing as a Separate Layer for an Animatable Human Avatar

We present SCANimate, an end-to-end trainable framework that takes raw 3D
scans of a clothed human and turns them into an animatable avatar. These
avatars are driven by pose parameters and have realistic clothing that moves
and deforms naturally. SCANimate does not rely on a customized mesh template or
surface mesh registration. We observe that fitting a parametric 3D body model,
like SMPL, to a clothed human scan is tractable while surface registration of
the body topology to the scan is often not, because clothing can deviate
significantly from the body shape. We also observe that articulated
transformations are invertible, resulting in geometric cycle consistency in the
posed and unposed shapes. These observations lead us to a weakly supervised
learning method that aligns scans into a canonical pose by disentangling
articulated deformations without template-based surface registration.
Furthermore, to complete missing regions in the aligned scans while modeling
pose-dependent deformations, we introduce a locally pose-aware implicit
function that learns to complete and model geometry with learned pose
correctives. In contrast to commonly used global pose embeddings, our local
pose conditioning significantly reduces long-range spurious correlations and
improves generalization to unseen poses, especially when training data is
limited. Our method can be applied to pose-aware appearance modeling to
generate a fully textured avatar. We demonstrate our approach on various
clothing types with different amounts of training data, outperforming existing
solutions and other variants in terms of fidelity and generality in every
setting. The code is available at this https URL

本文介绍了 SCANimate 系统，该系统采用弱监督学习方法，将原始三维扫描的拍穿人类转化为可动漫化的人物角色，在不需要表面网格配准的情况下实现装束的自然移动和变形，并引入了一种本地姿势感知隐式函数来模拟姿势依赖性形变。