Point-based representations have recently gained popularity in novel view
synthesis, for their unique advantages, e.g., intuitive geometric
representation, simple manipulation, and faster convergence. However, based on
our observation, these point-based neural re-rendering methods are only
expected to perform well under ideal conditions and suffer from noisy, patchy
points and unbounded scenes, which are challenging to handle but defacto common
in real applications. To this end, we revisit one such influential method,
known as Neural Point-based Graphics (NPBG), as our baseline, and propose
Robust Point-based Graphics (RPBG). We in-depth analyze the factors that
prevent NPBG from achieving satisfactory renderings on generic datasets, and
accordingly reform the pipeline to make it more robust to varying datasets
in-the-wild. Inspired by the practices in image restoration, we greatly enhance
the neural renderer to enable the attention-based correction of point
visibility and the inpainting of incomplete rasterization, with only acceptable
overheads. We also seek for a simple and lightweight alternative for
environment modeling and an iterative method to alleviate the problem of poor
geometry. By thorough evaluation on a wide range of datasets with different
shooting conditions and camera trajectories, RPBG stably outperforms the
baseline by a large margin, and exhibits its great robustness over
state-of-the-art NeRF-based variants. Code available at
this https URL

经深入分析 Neural Point-based Graphics (NPBG) 方法在通用数据集上无法实现令人满意的渲染效果，本研究提出了 Robust Point-based Graphics (RPBG) 方法，通过启发图像修复方法，大大增强了神经渲染器的能力，使其能够根据点的可见性进行修正，并补全不完整的光栅化，同时，通过简单轻量的环境建模和迭代方法，缓解了几何问题。经过对不同拍摄条件和摄像机轨迹的广泛评估，RPBG 在性能上明显优于基线方法，并展现了其对最新的 NeRF-based 变体的强大鲁棒性。