Neural radiance fields have achieved remarkable performance in modeling the
appearance of 3D scenes. However, existing approaches still struggle with the
view-dependent appearance of glossy surfaces, especially under complex lighting
of indoor environments. Unlike existing methods, which typically assume distant
lighting like an environment map, we propose a learnable Gaussian directional
encoding to better model the view-dependent effects under near-field lighting
conditions. Importantly, our new directional encoding captures the
spatially-varying nature of near-field lighting and emulates the behavior of
prefiltered environment maps. As a result, it enables the efficient evaluation
of preconvolved specular color at any 3D location with varying roughness
coefficients. We further introduce a data-driven geometry prior that helps
alleviate the shape radiance ambiguity in reflection modeling. We show that our
Gaussian directional encoding and geometry prior significantly improve the
modeling of challenging specular reflections in neural radiance fields, which
helps decompose appearance into more physically meaningful components.

神经辐射场在建模 3D 场景的外观方面取得了显著的性能，然而，现有方法在建模具有光泽表面的视角相关外观时仍存在困难，特别是在复杂照明的室内环境下。与现有方法不同的是，我们提出了一种可学习的高斯方向编码，以更好地模拟近场照明条件下的视角相关效果。重要的是，我们的新方向编码捕捉到近场照明的空间变化特性，并模拟预过滤环境贴图的行为。因此，它能够在具有不同粗糙度系数的任意 3D 位置有效评估预卷积的高光颜色。我们还引入了一种数据驱动的几何先验，帮助缓解反射建模中形状辐射的歧义。我们展示了我们的高斯方向编码和几何先验显著提高了神经辐射场中具有挑战性的高光反射的建模，从而将外观分解为更有物理含义的组成部分。

SpecNeRF: 光滑反射的高斯方向编码

SpecNeRF: Gaussian Directional Encoding for Specular Reflections

In this paper, we firstly consider view-dependent effects into single
image-based novel view synthesis (NVS) problems. For this, we propose to
exploit the camera motion priors in NVS to model view-dependent appearance or
effects (VDE) as the negative disparity in the scene. By recognizing
specularities "follow" the camera motion, we infuse VDEs into the input images
by aggregating input pixel colors along the negative depth region of the
epipolar lines. Also, we propose a `relaxed volumetric rendering' approximation
that allows computing the densities in a single pass, improving efficiency for
NVS from single images. Our method can learn single-image NVS from image
sequences only, which is a completely self-supervised learning method, for the
first time requiring neither depth nor camera pose annotations. We present
extensive experiment results and show that our proposed method can learn NVS
with VDEs, outperforming the SOTA single-view NVS methods on the RealEstate10k
and MannequinChallenge datasets.

本文首先考虑了单张基于图像的新视图合成（NVS）问题中的视角相关效果。为此，我们提出利用 NVS 中的相机运动先验来建模视角相关的外观或效果（VDE），即场景中的负视差。通过识别光斑 “跟随” 相机运动，我们通过在极线的负深度区域上聚合输入像素颜色来融入 VDE。同时，我们提出了一种 “放宽的体积渲染” 近似方法，可以在单次遍历中计算密度，提高了从单张图像进行 NVS 的效率。我们的方法只需从图像序列中学习单张图像 NVS，这是一种完全自我监督学习方法，首次无需深度或相机姿态注释。我们展示了广泛的实验证明，我们的方法可以学习具有 VDE 的 NVS，在 RealEstate10k 和 MannequinChallenge 数据集上优于 SOTA 的单视角 NVS 方法。