Neural radiance fields (NeRF) have revolutionized the field of image-based view synthesis. However, NeRF uses straight rays and fails to deal with complicated light path changes caused by refraction and reflection. This prevents NeRF from successfully synthesizing transparent or specular objects, which are ubiquitous in real-world robotics and A/VR applications. In this paper, we introduce the refractive-reflective field. Taking the object silhouette as input, we first utilize marching tetrahedra with a progressive encoding to reconstruct the geometry of non-Lambertian objects and then model refraction and reflection effects of the object in a unified framework using Fresnel terms. Meanwhile, to achieve efficient and effective anti-aliasing, we propose a virtual cone supersampling technique. We benchmark our method on different shapes, backgrounds and Fresnel terms on both real-world and synthetic datasets. We also qualitatively and quantitatively benchmark the rendering results of various editing applications, including material editing, object replacement/insertion, and environment illumination estimation. Codes and data are publicly available at https://github.com/dawning77/NeRRF.

本文介绍了一种新的方法，即折射-反射场，用于解决透明和镜面对象合成中的光线路径复杂性问题，并提出了一种虚拟锥超采样技术实现高效且有效的抗锯齿。通过在真实世界和合成数据集上进行测试，并对各种编辑应用进行定量和定性评估，包括材料编辑、对象替换/插入和环境照明估计，我们对该方法进行了基准测试。

透明和镜面对象的神经折射-反射场中的3D重建和视图合成