We present SplattingAvatar, a hybrid 3D representation of photorealistic
human avatars with Gaussian Splatting embedded on a triangle mesh, which
renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We
disentangle the motion and appearance of a virtual human with explicit mesh
geometry and implicit appearance modeling with Gaussian Splatting. The
Gaussians are defined by barycentric coordinates and displacement on a triangle
mesh as Phong surfaces. We extend lifted optimization to simultaneously
optimize the parameters of the Gaussians while walking on the triangle mesh.
SplattingAvatar is a hybrid representation of virtual humans where the mesh
represents low-frequency motion and surface deformation, while the Gaussians
take over the high-frequency geometry and detailed appearance. Unlike existing
deformation methods that rely on an MLP-based linear blend skinning (LBS) field
for motion, we control the rotation and translation of the Gaussians directly
by mesh, which empowers its compatibility with various animation techniques,
e.g., skeletal animation, blend shapes, and mesh editing. Trainable from
monocular videos for both full-body and head avatars, SplattingAvatar shows
state-of-the-art rendering quality across multiple datasets.

用高斯散焦渲染的三角网格中嵌入了高保真度人类虚拟角色的混合三维表示，可在现代 GPU 上达到每秒 300 帧，移动设备上达到每秒 30 帧，并通过显式网格几何和隐式外观模型化人物的运动与外观。

SplattingAvatar：基于网格嵌入高斯簇状点绘制的逼真实时人类头像

SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded  Gaussian Splatting

We introduce Neural Marching Cubes (NMC), a data-driven approach for
extracting a triangle mesh from a discretized implicit field. Classical MC is
defined by coarse tessellation templates isolated to individual cubes. While
more refined tessellations have been proposed, they all make heuristic
assumptions, such as trilinearity, when determining the vertex positions and
local mesh topologies in each cube. In principle, none of these approaches can
reconstruct geometric features that reveal coherence or dependencies between
nearby cubes (e.g., a sharp edge), as such information is unaccounted for,
resulting in poor estimates of the true underlying implicit field. To tackle
these challenges, we re-cast MC from a deep learning perspective, by designing
tessellation templates more apt at preserving geometric features, and learning
the vertex positions and mesh topologies from training meshes, to account for
contextual information from nearby cubes. We develop a compact per-cube
parameterization to represent the output triangle mesh, while being compatible
with neural processing, so that a simple 3D convolutional network can be
employed for the training. We show that all topological cases in each cube that
are applicable to our design can be easily derived using our representation,
and the resulting tessellations can also be obtained naturally and efficiently
by following a few design guidelines. In addition, our network learns local
features with limited receptive fields, hence it generalizes well to new shapes
and new datasets. We evaluate our neural MC approach by quantitative and
qualitative comparisons to all well-known MC variants. In particular, we
demonstrate the ability of our network to recover sharp features such as edges
and corners, a long-standing issue of MC and its variants. Our network also
reconstructs local mesh topologies more accurately than previous approaches.

本文提出了一种基于深度学习的 Neural Marching Cubes 方法，利用紧凑的每个立方体参数化来表示输出三角网格，并且是与神经处理兼容的，通过从训练网格中学习顶点位置和网格拓扑形状，以考虑周围立方体的上下文信息，从而更准确地重构几何特征和局部网格拓扑。

神经 Marching Cubes

Neural Marching Cubes

Rapid advances in 2D perception have led to systems that accurately detect
objects in real-world images. However, these systems make predictions in 2D,
ignoring the 3D structure of the world. Concurrently, advances in 3D shape
prediction have mostly focused on synthetic benchmarks and isolated objects. We
unify advances in these two areas. We propose a system that detects objects in
real-world images and produces a triangle mesh giving the full 3D shape of each
detected object. Our system, called Mesh R-CNN, augments Mask R-CNN with a mesh
prediction branch that outputs meshes with varying topological structure by
first predicting coarse voxel representations which are converted to meshes and
refined with a graph convolution network operating over the mesh's vertices and
edges. We validate our mesh prediction branch on ShapeNet, where we outperform
prior work on single-image shape prediction. We then deploy our full Mesh R-CNN
system on Pix3D, where we jointly detect objects and predict their 3D shapes.

该研究提出了一种称为 Mesh R-CNN 的系统，将 2D 感知和 3D 形状预测相结合，通过预测三角形网格的方式获取检测物体的完整 3D 形状，并在 Pix3D 上应用该系统来共同检测物体和预测它们的 3D 形状。

Mesh R-CNN

In this paper, we describe a robust algorithm for 2-Manifold generation of
various kinds of ShapeNet Models. The input of our pipeline is a triangle mesh,
with a set of vertices and triangular faces. The output of our pipeline is a
2-Manifold with vertices roughly uniformly distributed on the geometry surface.
Our algorithm uses an octree to represent the original mesh, and construct the
surface by isosurface extraction. Finally, we project the vertices to the
original mesh to achieve high precision. As a result, our method can be adopted
efficiently to all ShapeNet models with the guarantee of correct 2-Manifold
topology.

本文介绍了一种稳健的算法，用于生成各种 ShapeNet 模型的 2 流形。通过使用八叉树来表示原始网格，构建曲面并将顶点投影回原始网格，以实现高精度。