We present SplattingAvatar, a hybrid 3D representation of photorealistic
human avatars with Gaussian Splatting embedded on a triangle mesh, which
renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We
disentangle the motion and appearance of a virtual human with explicit mesh
geometry and implicit appearance modeling with Gaussian Splatting. The
Gaussians are defined by barycentric coordinates and displacement on a triangle
mesh as Phong surfaces. We extend lifted optimization to simultaneously
optimize the parameters of the Gaussians while walking on the triangle mesh.
SplattingAvatar is a hybrid representation of virtual humans where the mesh
represents low-frequency motion and surface deformation, while the Gaussians
take over the high-frequency geometry and detailed appearance. Unlike existing
deformation methods that rely on an MLP-based linear blend skinning (LBS) field
for motion, we control the rotation and translation of the Gaussians directly
by mesh, which empowers its compatibility with various animation techniques,
e.g., skeletal animation, blend shapes, and mesh editing. Trainable from
monocular videos for both full-body and head avatars, SplattingAvatar shows
state-of-the-art rendering quality across multiple datasets.

用高斯散焦渲染的三角网格中嵌入了高保真度人类虚拟角色的混合三维表示，可在现代 GPU 上达到每秒 300 帧，移动设备上达到每秒 30 帧，并通过显式网格几何和隐式外观模型化人物的运动与外观。

SplattingAvatar：基于网格嵌入高斯簇状点绘制的逼真实时人类头像

SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded  Gaussian Splatting

We introduce DMTet, a deep 3D conditional generative model that can
synthesize high-resolution 3D shapes using simple user guides such as coarse
voxels. It marries the merits of implicit and explicit 3D representations by
leveraging a novel hybrid 3D representation. Compared to the current implicit
approaches, which are trained to regress the signed distance values, DMTet
directly optimizes for the reconstructed surface, which enables us to
synthesize finer geometric details with fewer artifacts. Unlike deep 3D
generative models that directly generate explicit representations such as
meshes, our model can synthesize shapes with arbitrary topology. The core of
DMTet includes a deformable tetrahedral grid that encodes a discretized signed
distance function and a differentiable marching tetrahedra layer that converts
the implicit signed distance representation to the explicit surface mesh
representation. This combination allows joint optimization of the surface
geometry and topology as well as generation of the hierarchy of subdivisions
using reconstruction and adversarial losses defined explicitly on the surface
mesh. Our approach significantly outperforms existing work on conditional shape
synthesis from coarse voxel inputs, trained on a dataset of complex 3D animal
shapes. Project page: this https URL

介绍了一种深度三维条件生成模型 DMTet，它能够使用简单的用户指南（如粗略体素）合成高分辨率的三维形状。它将隐式和显式三维表示的优点结合起来，利用一种新颖的混合三维表示，并在表面网格上明确定义重建和对抗性损失，使我们能够综合更精细的几何细节，具有任意拓扑。