We present a new point-based approach for modeling the appearance of real
scenes. The approach uses a raw point cloud as the geometric representation of
a scene, and augments each point with a learnable neural descriptor that
encodes local geometry and appearance. A deep rendering network is learned in
parallel with the descriptors, so that new views of the scene can be obtained
by passing the rasterizations of a point cloud from new viewpoints through this
network. The input rasterizations use the learned descriptors as point
pseudo-colors. We show that the proposed approach can be used for modeling
complex scenes and obtaining their photorealistic views, while avoiding
explicit surface estimation and meshing. In particular, compelling results are
obtained for scene scanned using hand-held commodity RGB-D sensors as well as
standard RGB cameras even in the presence of objects that are challenging for
standard mesh-based modeling.

本研究提出了一种基于点的方法，使用可学习的神经描述符来编码局部几何和外观信息，并学习深度渲染网络生成具有照片般逼真感觉的场景视图，同时避免显式表面估计和网格化，并适用于各种复杂场景，包括使用 RGB-D 传感器与标准 RGB 相机进行扫描的场景。