Many 3D vision systems localize cameras within a scene using 3D point clouds.
Such point clouds are often obtained using structure from motion (SfM), after
which the images are discarded to preserve privacy. In this paper, we show, for
the first time, that such point clouds retain enough information to reveal
scene appearance and compromise privacy. We present a privacy attack that
reconstructs color images of the scene from the point cloud. Our method is
based on a cascaded U-Net that takes as input, a 2D multichannel image of the
points rendered from a specific viewpoint containing point depth and optionally
color and SIFT descriptors and outputs a color image of the scene from that
viewpoint. Unlike previous feature inversion methods, we deal with highly
sparse and irregular 2D point distributions and inputs where many point
attributes are missing, namely keypoint orientation and scale, the descriptor
image source and the 3D point visibility. We evaluate our attack algorithm on
public datasets and analyze the significance of the point cloud attributes.
Finally, we show that novel views can also be generated thereby enabling
compelling virtual tours of the underlying scene.

通过结构运动（SfM）算法获取的点云中的信息足以泄露场景外貌并危及隐私，同时使用级联 U-Net 方法可以重构场景的彩色图像，并生成新的视图，为场景的虚拟旅游提供了可能。