We develop an effective point cloud rendering pipeline for novel view
synthesis, which enables high fidelity local detail reconstruction, real-time
rendering and user-friendly editing. In the heart of our pipeline is an
adaptive frequency modulation module called Adaptive Frequency Net (AFNet),
which utilizes a hypernetwork to learn the local texture frequency encoding
that is consecutively injected into adaptive frequency activation layers to
modulate the implicit radiance signal. This mechanism improves the frequency
expressive ability of the network with richer frequency basis support, only at
a small computational budget. To further boost performance, a preprocessing
module is also proposed for point cloud geometry optimization via point opacity
estimation. In contrast to implicit rendering, our pipeline supports
high-fidelity interactive editing based on point cloud manipulation. Extensive
experimental results on NeRF-Synthetic, ScanNet, DTU and Tanks and Temples
datasets demonstrate the superior performances achieved by our method in terms
of PSNR, SSIM and LPIPS, in comparison to the state-of-the-art.

本文提出了一种适用于新视角合成的有效的点云渲染管道，其中包括自适应频率调制模块，用以提供更丰富的频率支持并改善网络性能，并支持高保真度交互式编辑。实验结果也表明本方法在性能表现方面优于现有最先进技术。

使用简单编辑的频率调制点云渲染

Frequency-Modulated Point Cloud Rendering with Easy Editing

We present a new deep point cloud rendering pipeline through multi-plane
projections. The input to the network is the raw point cloud of a scene and the
output are image or image sequences from a novel view or along a novel camera
trajectory. Unlike previous approaches that directly project features from 3D
points onto 2D image domain, we propose to project these features into a
layered volume of camera frustum. In this way, the visibility of 3D points can
be automatically learnt by the network, such that ghosting effects due to false
visibility check as well as occlusions caused by noise interferences are both
avoided successfully. Next, the 3D feature volume is fed into a 3D CNN to
produce multiple layers of images w.r.t. the space division in the depth
directions. The layered images are then blended based on learned weights to
produce the final rendering results. Experiments show that our network produces
more stable renderings compared to previous methods, especially near the object
boundaries. Moreover, our pipeline is robust to noisy and relatively sparse
point cloud for a variety of challenging scenes.

本研究提出了一种新的深度学习点云渲染管道，通过多平面投影将点云特征投射到摄像机视锥体的分层体积中，然后通过卷积神经网络产生多层图像，并根据学习到的权重混合图像，最终实现对具有挑战的场景进行更稳定的渲染。