Point clouds captured by scanning devices are often incomplete due to
occlusion. Point cloud completion aims to predict the complete shape based on
its partial input. Existing methods can be classified into supervised and
unsupervised methods. However, both of them require a large number of 3D
complete point clouds, which are difficult to capture. In this paper, we
propose Cross-PCC, an unsupervised point cloud completion method without
requiring any 3D complete point clouds. We only utilize 2D images of the
complete objects, which are easier to capture than 3D complete and clean point
clouds. Specifically, to take advantage of the complementary information from
2D images, we use a single-view RGB image to extract 2D features and design a
fusion module to fuse the 2D and 3D features extracted from the partial point
cloud. To guide the shape of predicted point clouds, we project the predicted
points of the object to the 2D plane and use the foreground pixels of its
silhouette maps to constrain the position of the projected points. To reduce
the outliers of the predicted point clouds, we propose a view calibrator to
move the points projected to the background into the foreground by the
single-view silhouette image. To the best of our knowledge, our approach is the
first point cloud completion method that does not require any 3D supervision.
The experimental results of our method are superior to those of the
state-of-the-art unsupervised methods by a large margin. Moreover, compared to
some supervised methods, our method achieves similar performance. We will make
the source code publicly available at this https URL

本文提出了一种无监督的点云补全方法，不需要 3D 补全点云，只利用 2D 补全图像，使用单视角 RGB 图像提取 2D 特征，并使用融合模块将其与部分点云提取的 3D 特征相结合，以预测物体的完整形状。

利用单视图图像进行无监督三维点云补全

Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion

Indoor scene semantic parsing from RGB images is very challenging due to
occlusions, object distortion, and viewpoint variations. Going beyond prior
works that leverage geometry information, typically paired depth maps, we
present a new approach, a 3D-to-2D distillation framework, that enables us to
leverage 3D features extracted from large-scale 3D data repository (e.g.,
ScanNet-v2) to enhance 2D features extracted from RGB images. Our work has
three novel contributions. First, we distill 3D knowledge from a pretrained 3D
network to supervise a 2D network to learn simulated 3D features from 2D
features during the training, so the 2D network can infer without requiring 3D
data. Second, we design a two-stage dimension normalization scheme to calibrate
the 2D and 3D features for better integration. Third, we design a
semantic-aware adversarial training model to extend our framework for training
with unpaired 3D data. Extensive experiments on various datasets, ScanNet-V2,
S3DIS, and NYU-v2, demonstrate the superiority of our approach. Also,
experimental results show that our 3D-to-2D distillation improves the model
generalization.

该论文提出了一种新的室内场景语义分割方法，使用一个新的 3D-to-2D 分解框架，可以从大规模的 3D 数据中提取 3D 特征，以增强从 RGB 图像中提取的 2D 特征，并通过对其进行标准化和语义相关的对抗性训练对其进行改进。