We have recently seen great progress in building photorealistic animatable
full-body codec avatars, but generating high-fidelity animation of clothing is
still difficult. To address these difficulties, we propose a method to build an
animatable clothed body avatar with an explicit representation of the clothing
on the upper body from multi-view captured videos. We use a two-layer mesh
representation to register each 3D scan separately with the body and clothing
templates. In order to improve the photometric correspondence across different
frames, texture alignment is then performed through inverse rendering of the
clothing geometry and texture predicted by a variational autoencoder. We then
train a new two-layer codec avatar with separate modeling of the upper clothing
and the inner body layer. To learn the interaction between the body dynamics
and clothing states, we use a temporal convolution network to predict the
clothing latent code based on a sequence of input skeletal poses. We show
photorealistic animation output for three different actors, and demonstrate the
advantage of our clothed-body avatars over the single-layer avatars used in
previous work. We also show the benefit of an explicit clothing model that
allows the clothing texture to be edited in the animation output.

通过多视角捕捉视频对具有显式服装表示的可动衣物人形头像进行建模，采用二层网格表示注册每个 3D 扫描，通过时序卷积网络来预测衣物潜在编码以实现人体动力学和服装状态的交互作用。