In recent years, sparse voxel-based methods have become the state-of-the-arts
for 3D semantic segmentation of indoor scenes, thanks to the powerful 3D CNNs.
Nevertheless, being oblivious to the underlying geometry, voxel-based methods
suffer from ambiguous features on spatially close objects and struggle with
handling complex and irregular geometries due to the lack of geodesic
information. In view of this, we present Voxel-Mesh Network (VMNet), a novel 3D
deep architecture that operates on the voxel and mesh representations
leveraging both the Euclidean and geodesic information. Intuitively, the
Euclidean information extracted from voxels can offer contextual cues
representing interactions between nearby objects, while the geodesic
information extracted from meshes can help separate objects that are spatially
close but have disconnected surfaces. To incorporate such information from the
two domains, we design an intra-domain attentive module for effective feature
aggregation and an inter-domain attentive module for adaptive feature fusion.
Experimental results validate the effectiveness of VMNet: specifically, on the
challenging ScanNet dataset for large-scale segmentation of indoor scenes, it
outperforms the state-of-the-art SparseConvNet and MinkowskiNet (74.6% vs 72.5%
and 73.6% in mIoU) with a simpler network structure (17M vs 30M and 38M
parameters). Code release: this https URL

本篇论文介绍了一种基于体素和网格表达，利用欧几里得信息和测地信息的深度学习体系结构 VMNet 用于室内场景的 3D 语义分割，并在实验证明了其优越性能，尤其是在大规模的 ScanNet 数据集上。