Face reenactment is challenging due to the need to establish dense correspondence between various face representations for motion transfer. Recent studies have utilized Neural Radiance Field (NeRF) as fundamental representation, which further enhanced the performance of multi-view face reenactment in photo-realism and 3D consistency. However, establishing dense correspondence between different face NeRFs is non-trivial, because implicit representations lack ground-truth correspondence annotations like mesh-based 3D parametric models (e.g., 3DMM) with index-aligned vertexes. Although aligning 3DMM space with NeRF-based face representations can realize motion control, it is sub-optimal for their limited face-only modeling and low identity fidelity. Therefore, we are inspired to ask: Can we learn the dense correspondence between different NeRF-based face representations without a 3D parametric model prior? To address this challenge, we propose a novel framework, which adopts tri-planes as fundamental NeRF representation and decomposes face tri-planes into three components: canonical tri-planes, identity deformations, and motion. In terms of motion control, our key contribution is proposing a Plane Dictionary (PlaneDict) module, which efficiently maps the motion conditions to a linear weighted addition of learnable orthogonal plane bases. To the best of our knowledge, our framework is the first method that achieves one-shot multi-view face reenactment without a 3D parametric model prior. Extensive experiments demonstrate that we produce better results in fine-grained motion control and identity preservation than previous methods.

我们提出了一种新颖的框架，使用三面体平面作为基本NeRF表示，并将面的三个部分分解为：规范三面体平面、身份形变和运动。在运动控制方面，我们的主要贡献是提出了一个平面字典（PlaneDict）模块，它将运动条件高效地映射到可学习的正交平面基的线性加权相加。据我们所知，我们的框架是第一个在没有3D参数模型先验的情况下实现一次性多视角面部重现的方法。大量实验表明，我们在细粒度运动控制和身份保持方面产生了比以前的方法更好的结果。

基于 NeRF 的人脸再现的密集对应关系学习