Creating controllable, photorealistic, and geometrically detailed digital
doubles of real humans solely from video data is a key challenge in Computer
Graphics and Vision, especially when real-time performance is required. Recent
methods attach a neural radiance field (NeRF) to an articulated structure,
e.g., a body model or a skeleton, to map points into a pose canonical space
while conditioning the NeRF on the skeletal pose. These approaches typically
parameterize the neural field with a multi-layer perceptron (MLP) leading to a
slow runtime. To address this drawback, we propose TriHuman a novel
human-tailored, deformable, and efficient tri-plane representation, which
achieves real-time performance, state-of-the-art pose-controllable geometry
synthesis as well as photorealistic rendering quality. At the core, we
non-rigidly warp global ray samples into our undeformed tri-plane texture
space, which effectively addresses the problem of global points being mapped to
the same tri-plane locations. We then show how such a tri-plane feature
representation can be conditioned on the skeletal motion to account for dynamic
appearance and geometry changes. Our results demonstrate a clear step towards
higher quality in terms of geometry and appearance modeling of humans as well
as runtime performance.

通过视频数据创建可控、逼真、几何详细的数字化人类复制品是计算机图形学和视觉领域的关键挑战，特别是当需要实时性能时。最近的方法将神经光辐射场（NeRF）附加到关节结构上，如身体模型或骨骼，将点映射到姿势规范空间，同时条件化 NeRF 的骨骼姿势。我们提出了 TriHuman，一个新的人体定制、可变形和高效的三平面表示方法，实现了实时性能、最先进的姿势控制几何合成以及逼真的渲染质量。在核心部分中，我们非刚性地将全局光线采样变形到无变形的三平面纹理空间中，这有效解决了将全局点映射到相同三平面位置的问题。然后，我们展示了如何将这种三平面特征表示方法与骨骼运动相关联，以考虑动态外观和几何变化。我们的结果表明，对于人体的几何和外观建模以及运行时间性能，有了明显的质量提升。