3D reconstruction and simulation, while interrelated, have distinct
objectives: reconstruction demands a flexible 3D representation adaptable to
diverse scenes, whereas simulation requires a structured representation to
model motion principles effectively. This paper introduces the Mesh-adsorbed
Gaussian Splatting (MaGS) method to resolve such a dilemma. MaGS constrains 3D
Gaussians to hover on the mesh surface, creating a mutual-adsorbed
mesh-Gaussian 3D representation that combines the rendering flexibility of 3D
Gaussians with the spatial coherence of meshes. Leveraging this representation,
we introduce a learnable Relative Deformation Field (RDF) to model the relative
displacement between the mesh and 3D Gaussians, extending traditional
mesh-driven deformation paradigms that only rely on ARAP prior, thus capturing
the motion of each 3D Gaussian more precisely. By joint optimizing meshes, 3D
Gaussians, and RDF, MaGS achieves both high rendering accuracy and realistic
deformation. Extensive experiments on the D-NeRF and NeRF-DS datasets
demonstrate that MaGS can generate competitive results in both reconstruction
and simulation.

本文介绍了 Mesh-adsorbed Gaussian Splatting（MaGS）方法，它通过在网格表面生成互相吸附的网格 - 高斯三维表示，结合了三维高斯的渲染灵活性和网格的空间一致性，引入了可学习的 Relative Deformation Field（RDF）来更精确地捕捉每个三维高斯的运动，从而实现了高渲染准确性和逼真变形，在 D-NeRF 和 NeRF-DS 数据集上的实验表明，MaGS 在重建和模拟方面取得了竞争性的结果。