Pixel2Mesh (P2M) is a classical approach for reconstructing 3D shapes from a
single color image through coarse-to-fine mesh deformation. Although P2M is
capable of generating plausible global shapes, its Graph Convolution Network
(GCN) often produces overly smooth results, causing the loss of fine-grained
geometry details. Moreover, P2M generates non-credible features for occluded
regions and struggles with the domain gap from synthetic data to real-world
images, which is a common challenge for single-view 3D reconstruction methods.
To address these challenges, we propose a novel Transformer-boosted
architecture, named T-Pixel2Mesh, inspired by the coarse-to-fine approach of
P2M. Specifically, we use a global Transformer to control the holistic shape
and a local Transformer to progressively refine the local geometry details with
graph-based point upsampling. To enhance real-world reconstruction, we present
the simple yet effective Linear Scale Search (LSS), which serves as prompt
tuning during the input preprocessing. Our experiments on ShapeNet demonstrate
state-of-the-art performance, while results on real-world data show the
generalization capability.

基于传统的像素到网格变形方法 Pixel2Mesh，本文提出了一种名为 T-Pixel2Mesh 的新型转换增强架构，通过全局 Transformer 控制整体形状和局部 Transformer 逐步细化局部几何细节，结合基于图的点上采样，在真实世界的数据上展示了最先进的性能和普适性能力。