Controllability, generalizability and efficiency are the major objectives of
constructing face avatars represented by neural implicit field. However,
existing methods have not managed to accommodate the three requirements
simultaneously. They either focus on static portraits, restricting the
representation ability to a specific subject, or suffer from substantial
computational cost, limiting their flexibility. In this paper, we propose
One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a
generalized controllable tri-plane rendering solution so that each personalized
avatar can be constructed from only one portrait as the reference.
Specifically, OTAvatar first inverts a portrait image to a motion-free identity
code. Second, the identity code and a motion code are utilized to modulate an
efficient CNN to generate a tri-plane formulated volume, which encodes the
subject in the desired motion. Finally, volume rendering is employed to
generate an image in any view. The core of our solution is a novel
decoupling-by-inverting strategy that disentangles identity and motion in the
latent code via optimization-based inversion. Benefiting from the efficient
tri-plane representation, we achieve controllable rendering of generalized face
avatar at $35$ FPS on A100. Experiments show promising performance of
cross-identity reenactment on subjects out of the training set and better 3D
consistency.

本研究提出了一种基于广义控制的三平面渲染方案构建面部头像的方法 ——OTAvatar，通过利用运动代码和个体身份编码以及体积渲染生成的体积，实现了通用面部头像的可控渲染，并在 A100 达到了 35FPS 的效率。