This paper introduces a novel approach for 3D semantic instance segmentation
on point clouds. A 3D convolutional neural network called submanifold sparse
convolutional network is used to generate semantic predictions and instance
embeddings simultaneously. To obtain discriminative embeddings for each 3D
instance, a structure-aware loss function is proposed which considers both the
structure information and the embedding information. To get more consistent
embeddings for each 3D instance, attention-based k nearest neighbour (KNN) is
proposed to assign different weights for different neighbours. Based on the
attention-based KNN, we add a graph convolutional network after the sparse
convolutional network to get refined embeddings. Our network can be trained
end-to-end. A simple mean-shift algorithm is utilized to cluster refined
embeddings to get final instance predictions. As a result, our framework can
output both the semantic prediction and the instance prediction. Experiments
show that our approach outperforms all state-of-art methods on ScanNet
benchmark and NYUv2 dataset.

本论文提出了一种新的 3D 语义实例分割方法，利用子流形稀疏卷积网络同时生成语义预测和实例嵌入，通过结构感知的损失函数和基于关注度的 k 最近邻算法获取区分嵌入，加入图卷积网络以获取精细嵌入并输出语义和实例预测，实验结果表明该方法在 ScanNet 基准和 NYUv2 数据集上优于所有最先进方法.