Neural Radiance Fields (NeRF) is a revolutionary approach for rendering
scenes by sampling a single ray per pixel and it has demonstrated impressive
capabilities in novel-view synthesis from static scene images. However, in
practice, we usually need to recover NeRF from unconstrained image collections,
which poses two challenges: 1) the images often have dynamic changes in
appearance because of different capturing time and camera settings; 2) the
images may contain transient objects such as humans and cars, leading to
occlusion and ghosting artifacts. Conventional approaches seek to address these
challenges by locally utilizing a single ray to synthesize a color of a pixel.
In contrast, humans typically perceive appearance and objects by globally
utilizing information across multiple pixels. To mimic the perception process
of humans, in this paper, we propose Cross-Ray NeRF (CR-NeRF) that leverages
interactive information across multiple rays to synthesize occlusion-free novel
views with the same appearances as the images. Specifically, to model varying
appearances, we first propose to represent multiple rays with a novel cross-ray
feature and then recover the appearance by fusing global statistics, i.e.,
feature covariance of the rays and the image appearance. Moreover, to avoid
occlusion introduced by transient objects, we propose a transient objects
handler and introduce a grid sampling strategy for masking out the transient
objects. We theoretically find that leveraging correlation across multiple rays
promotes capturing more global information. Moreover, extensive experimental
results on large real-world datasets verify the effectiveness of CR-NeRF.

本文提出了一种名为 Cross-Ray NeRF 的方法，通过利用交互信息合成视野，在处理过场景图像时能有效地减少遮挡和幽灵物体等问题。