Learning per-point semantic features from the hierarchical feature pyramid is
essential for point cloud semantic segmentation. However, most previous methods
suffered from ambiguous region features or failed to refine per-point features
effectively, which leads to information loss and ambiguous semantic
identification. To resolve this, we propose Retro-FPN to model the per-point
feature prediction as an explicit and retrospective refining process, which
goes through all the pyramid layers to extract semantic features explicitly for
each point. Its key novelty is a retro-transformer for summarizing semantic
contexts from the previous layer and accordingly refining the features in the
current stage. In this way, the categorization of each point is conditioned on
its local semantic pattern. Specifically, the retro-transformer consists of a
local cross-attention block and a semantic gate unit. The cross-attention
serves to summarize the semantic pattern retrospectively from the previous
layer. And the gate unit carefully incorporates the summarized contexts and
refines the current semantic features. Retro-FPN is a pluggable neural network
that applies to hierarchical decoders. By integrating Retro-FPN with three
representative backbones, including both point-based and voxel-based methods,
we show that Retro-FPN can significantly improve performance over
state-of-the-art backbones. Comprehensive experiments on widely used benchmarks
can justify the effectiveness of our design. The source is available at
this https URL

通过在金字塔层面上建模每个点的特征预测，并利用回溯式的精炼过程从前一层中提取语义特征，本文提出 Retro-FPN 方法以解决点云语义分割中的信息丢失和模糊语义识别的问题，该方法在三个常用的骨干网络上得到了显著的性能提升。