Monocular scene reconstruction from posed images is challenging due to the
complexity of a large environment. Recent volumetric methods learn to directly
predict the TSDF volume and have demonstrated promising results in this task.
However, most methods focus on how to extract and fuse the 2D features to a 3D
feature volume, but none of them improve the way how the 3D volume is
aggregated. In this work, we propose an SDF transformer network, which replaces
the role of 3D CNN for better 3D feature aggregation. To reduce the explosive
computation complexity of the 3D multi-head attention, we propose a sparse
window attention module, where the attention is only calculated between the
non-empty voxels within a local window. Then a top-down-bottom-up 3D attention
network is built for 3D feature aggregation, where a dilate-attention structure
is proposed to prevent geometry degeneration, and two global modules are
employed to equip with global receptive fields. The experiments on multiple
datasets show that this 3D transformer network generates a more accurate and
complete reconstruction, which outperforms previous methods by a large margin.
Remarkably, the mesh accuracy is improved by 41.8%, and the mesh completeness
is improved by 25.3% on the ScanNet dataset. Project page:
this https URL.

本文提出了一种基于 SDF 转换网络、稀疏窗口注意力模块和上下文注意力网络的三维变换器网络，用于更好地聚合三维特征，提高单目场景重建的精度和完整性。实验结果表明，该网络在多个数据集上优于现有方法，能够提高网格的准确性和完整性。