This paper presents Neural Mesh Fusion (NMF), an efficient approach for joint
optimization of polygon mesh from multi-view image observations and
unsupervised 3D planar-surface parsing of the scene. In contrast to implicit
neural representations, NMF directly learns to deform surface triangle mesh and
generate an embedding for unsupervised 3D planar segmentation through
gradient-based optimization directly on the surface mesh. The conducted
experiments show that NMF obtains competitive results compared to
state-of-the-art multi-view planar reconstruction, while not requiring any
ground-truth 3D or planar supervision. Moreover, NMF is significantly more
computationally efficient compared to implicit neural rendering-based scene
reconstruction approaches.

该研究提出了神经网格融合（NMF），这是一个高效的方法，用于联合优化多视角图像观测的多边形网格和场景的无监督 3D 平面分割。与隐式神经表示不同，NMF 直接学习变形表面三角网格，并通过基于梯度的优化在表面网格上生成无监督 3D 平面分割的嵌入。实验证明，与最先进的多视角平面重建方法相比，NMF 获得了有竞争力的结果，同时不需要任何 3D 或平面的真实监督。此外，与基于隐式神经渲染的场景重建方法相比，NMF 在计算上更加高效。

神经网络网格融合：无监督的三维平面表面理解

Neural Mesh Fusion: Unsupervised 3D Planar Surface Understanding

Monocular 3D reconstruction is to reconstruct the shape of object and its
other information from a single RGB image. In 3D reconstruction, polygon mesh,
with detailed surface information and low computational cost, is the most
prevalent expression form obtained from deep learning models. However, the
state-of-the-art schemes fail to directly generate well-structured meshes, and
we identify that most meshes have severe Vertices Clustering (VC) and Illegal
Twist (IT) problems. By analyzing the mesh deformation process, we pinpoint
that the inappropriate usage of Chamfer Distance (CD) loss is a root cause of
VC and IT problems in deep learning model. In this paper, we initially
demonstrate these two problems induced by CD loss with visual examples and
quantitative analyses. Then, we propose a fine-grained reconstruction method
CD$^2$ by employing Chamfer distance twice to perform a plausible and adaptive
deformation. Extensive experiments on two 3D datasets and comparisons with five
latest schemes demonstrate that our CD$^2$ directly generates a well-structured
mesh and outperforms others in terms of several quantitative metrics.

本文提出通过两次使用 Chamfer 距离来进行适当的变形，从而直接生成结构良好的网格的方法 CD$^2$，并在两个 3D 数据集上进行实验证明其在多个定量指标上优于其他方法。

CD$^2$: 微观的、 细致的三维网格重建，具有两倍的 距离切除度量值

CD$^2$: Fine-grained 3D Mesh Reconstruction With Twice Chamfer Distance

For modeling the 3D world behind 2D images, which 3D representation is most
appropriate? A polygon mesh is a promising candidate for its compactness and
geometric properties. However, it is not straightforward to model a polygon
mesh from 2D images using neural networks because the conversion from a mesh to
an image, or rendering, involves a discrete operation called rasterization,
which prevents back-propagation. Therefore, in this work, we propose an
approximate gradient for rasterization that enables the integration of
rendering into neural networks. Using this renderer, we perform single-image 3D
mesh reconstruction with silhouette image supervision and our system
outperforms the existing voxel-based approach. Additionally, we perform
gradient-based 3D mesh editing operations, such as 2D-to-3D style transfer and
3D DeepDream, with 2D supervision for the first time. These applications
demonstrate the potential of the integration of a mesh renderer into neural
networks and the effectiveness of our proposed renderer.

本文介绍了一种基于神经网络的多边形网格渲染方法，该方法使用了一种近似梯度来克服离散化操作的限制，从而实现了单幅图像的 3D 网格重建。同时，该方法还实现了基于 2D 监督的 3D 渲染编辑操作。实验结果表明，所提出的渲染器有效性强。