Talking head generation based on the neural radiation fields model has shown
promising visual effects. However, the slow rendering speed of NeRF seriously
limits its application, due to the burdensome calculation process over hundreds
of sampled points to synthesize one pixel. In this work, a novel Neural Light
Dynamic Fields model is proposed aiming to achieve generating high quality 3D
talking face with significant speedup. The NLDF represents light fields based
on light segments, and a deep network is used to learn the entire light beam's
information at once. In learning the knowledge distillation is applied and the
NeRF based synthesized result is used to guide the correct coloration of light
segments in NLDF. Furthermore, a novel active pool training strategy is
proposed to focus on high frequency movements, particularly on the speaker
mouth and eyebrows. The propose method effectively represents the facial light
dynamics in 3D talking video generation, and it achieves approximately 30 times
faster speed compared to state of the art NeRF based method, with comparable
generation visual quality.

提出了一种基于神经辐射场模型的说话脸生成方法，通过利用神经光动力场模型来加速渲染速度，同时使用知识蒸馏和动态训练策略，实现了生成高质量三维说话脸视频的效果和速度的显著提升。