Learning to generate 3D point clouds without 3D supervision is an important
but challenging problem. Current solutions leverage various differentiable
renderers to project the generated 3D point clouds onto a 2D image plane, and
train deep neural networks using the per-pixel difference with 2D ground truth
images. However, these solutions are still struggling to fully recover fine
structures of 3D shapes, such as thin tubes or planes. To resolve this issue,
we propose an unsupervised approach for 3D point cloud generation with fine
structures. Specifically, we cast 3D point cloud learning as a 2D projection
matching problem. Rather than using entire 2D silhouette images as a regular
pixel supervision, we introduce structure adaptive sampling to randomly sample
2D points within the silhouettes as an irregular point supervision, which
alleviates the consistency issue of sampling from different view angles. Our
method pushes the neural network to generate a 3D point cloud whose 2D
projections match the irregular point supervision from different view angles.
Our 2D projection matching approach enables the neural network to learn more
accurate structure information than using the per-pixel difference, especially
for fine and thin 3D structures. Our method can recover fine 3D structures from
2D silhouette images at different resolutions, and is robust to different
sampling methods and point number in irregular point supervision. Our method
outperforms others under widely used benchmarks. Our code, data and models are
available at this https URL

提出了一种无监督方法来生成带有细微结构的 3D 点云，使用自适应采样来不规则地采样 2D 轮廓图像中的点，从而解决不同视角采样的一致性问题，使用 2D 投影匹配方法和深度神经网络，学习精确的结构信息，从而使其能够恢复细节丰富的 3D 结构，其表现优于其他相关基准算法。