Semantic scene understanding from point clouds is particularly challenging as
the points reflect only a sparse set of the underlying 3D geometry. Previous
works often convert point cloud into regular grids (e.g. voxels or bird-eye
view images), and resort to grid-based convolutions for scene understanding. In
this work, we introduce RfD-Net that jointly detects and reconstructs dense
object surfaces directly from raw point clouds. Instead of representing scenes
with regular grids, our method leverages the sparsity of point cloud data and
focuses on predicting shapes that are recognized with high objectness. With
this design, we decouple the instance reconstruction into global object
localization and local shape prediction. It not only eases the difficulty of
learning 2-D manifold surfaces from sparse 3D space, the point clouds in each
object proposal convey shape details that support implicit function learning to
reconstruct any high-resolution surfaces. Our experiments indicate that
instance detection and reconstruction present complementary effects, where the
shape prediction head shows consistent effects on improving object detection
with modern 3D proposal network backbones. The qualitative and quantitative
evaluations further demonstrate that our approach consistently outperforms the
state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.

本研究介绍了 Rfd-Net，该方法通过在原始点云数据中直接检测和重构密集对象表面，摆脱了使用常规网格表示场景的限制，旨在预测可识别高对象性的形状，从而实现全局对象定位和局部形状预测，提高了物体检测的精度，比现有技术研究结果提高了 11 个三维网格交并比。