Recent communities have seen significant progress in building photo-realistic
animatable avatars from sparse multi-view videos. However, current workflows
struggle to render realistic garment dynamics for loose-fitting characters as
they predominantly rely on naked body models for human modeling while leaving
the garment part un-modeled. This is mainly due to that the deformations
yielded by loose garments are highly non-rigid, and capturing such deformations
often requires dense views as supervision. In this paper, we introduce
AniDress, a novel method for generating animatable human avatars in loose
clothes using very sparse multi-view videos (4-8 in our setting). To allow the
capturing and appearance learning of loose garments in such a situation, we
employ a virtual bone-based garment rigging model obtained from physics-based
simulation data. Such a model allows us to capture and render complex garment
dynamics through a set of low-dimensional bone transformations. Technically, we
develop a novel method for estimating temporal coherent garment dynamics from a
sparse multi-view video. To build a realistic rendering for unseen garment
status using coarse estimations, a pose-driven deformable neural radiance field
conditioned on both body and garment motions is introduced, providing explicit
control of both parts. At test time, the new garment poses can be captured from
unseen situations, derived from a physics-based or neural network-based
simulator to drive unseen garment dynamics. To evaluate our approach, we create
a multi-view dataset that captures loose-dressed performers with diverse
motions. Experiments show that our method is able to render natural garment
dynamics that deviate highly from the body and generalize well to both unseen
views and poses, surpassing the performance of existing methods. The code and
data will be publicly available.

在本文中，我们介绍了一种使用非常稀疏的多视角视频（我们的设置中为 4-8 个）生成穿着宽松服装的可动画人类化身的新方法 AniDress。为了在这种情况下捕捉和学习宽松服装的外观，我们采用从基于物理的模拟数据中获得的虚拟基于骨骼的服装绑定模型。通过一组低维骨骼变换，这种模型使我们能够捕捉和渲染复杂的服装动态。技术上，我们开发了一种从稀疏多视角视频中估计时间一致服装动态的新方法。为了利用粗略估计建立逼真的未见状态渲染，我们引入了一个在身体和服装运动上都受条件约束的基于姿势驱动的可变形神经辐射场，提供了两部分的显式控制。实验证明，我们的方法能够呈现高度偏离身体的自然服装动态，并且在未见视角和姿势上具有良好的泛化性能，超过现有方法的表现。代码和数据将公开提供。

AniDress: 通过稀疏视图使用服装绑定模型制作可动画的宽松裙装化身

AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using  Garment Rigging Model

A vital task of the wider digital human effort is the creation of realistic
garments on digital avatars, both in the form of characteristic fold patterns
and wrinkles in static frames as well as richness of garment dynamics under
avatars' motion. Existing workflow of modeling, simulation, and rendering
closely replicates the physics behind real garments, but is tedious and
requires repeating most of the workflow under changes to characters' motion,
camera angle, or garment resizing. Although data-driven solutions exist, they
either focus on static scenarios or only handle dynamics of tight garments. We
present a solution that, at test time, takes in body joint motion to directly
produce realistic dynamic garment image sequences. Specifically, given the
target joint motion sequence of an avatar, we propose dynamic neural garments
to jointly simulate and render plausible dynamic garment appearance from an
unseen viewpoint. Technically, our solution generates a coarse garment proxy
sequence, learns deep dynamic features attached to this template, and neurally
renders the features to produce appearance changes such as folds, wrinkles, and
silhouettes. We demonstrate generalization behavior to both unseen motion and
unseen camera views. Further, our network can be fine-tuned to adopt to new
body shape and/or background images. We also provide comparisons against
existing neural rendering and image sequence translation approaches, and report
clear quantitative improvements.

本文介绍了一种解决数字化人物着装的解决方案，该方案可以通过神经网络在测试时直接生成逼真的动态服装图像序列，以解决现有工作流程中重复性高且耗时的问题，并且可以在新的身体形状和背景图像下进行微调，并提供了与现有神经渲染和图像序列转换方法的定量比较。