The advent of neural and Gaussian-based radiance field methods have achieved
great success in the field of novel view synthesis. However, specular
reflection remains non-trivial, as the high frequency radiance field is
notoriously difficult to fit stably and accurately. We present a deferred
shading method to effectively render specular reflection with Gaussian
splatting. The key challenge comes from the environment map reflection model,
which requires accurate surface normal while simultaneously bottlenecks normal
estimation with discontinuous gradients. We leverage the per-pixel reflection
gradients generated by deferred shading to bridge the optimization process of
neighboring Gaussians, allowing nearly correct normal estimations to gradually
propagate and eventually spread over all reflective objects. Our method
significantly outperforms state-of-the-art techniques and concurrent work in
synthesizing high-quality specular reflection effects, demonstrating a
consistent improvement of peak signal-to-noise ratio (PSNR) for both synthetic
and real-world scenes, while running at a frame rate almost identical to
vanilla Gaussian splatting.

基于神经和高斯辐射场方法，在小说视角合成领域取得了巨大的成功。然而，镜面反射仍然非常复杂，因为高频辐射场非常难以稳定准确地拟合。我们提出了一种延迟着色方法来有效地渲染具有高斯分片的镜面反射。主要挑战来自环境映射反射模型，该模型要求精确的表面法线，同时在不连续梯度处瓶颈法线估计。我们利用延迟着色生成的每像素反射梯度来桥接邻域高斯优化过程，使几乎准确的法线估计逐渐传播并最终传播到所有反射物体。我们的方法在合成和真实场景中，显著优于最先进的技术和同时进行的工作，表现出峰值信噪比（PSNR）的一致提高，同时运行速度几乎与原始高斯分片相同。

3D 高斯点刷及延迟反射

3D Gaussian Splatting with Deferred Reflection

Reconstructing the 3D shape of an object using several images under different
light sources is a very challenging task, especially when realistic assumptions
such as light propagation and attenuation, perspective viewing geometry and
specular light reflection are considered. Many of works tackling Photometric
Stereo (PS) problems often relax most of the aforementioned assumptions.
Especially they ignore specular reflection and global illumination effects. In
this work, we propose the first CNN based approach capable of handling these
realistic assumptions in Photometric Stereo. We leverage recent improvements of
deep neural networks for far-field Photometric Stereo and adapt them to near
field setup. We achieve this by employing an iterative procedure for shape
estimation which has two main steps. Firstly we train a per-pixel CNN to
predict surface normals from reflectance samples. Secondly, we compute the
depth by integrating the normal field in order to iteratively estimate light
directions and attenuation which is used to compensate the input images to
compute reflectance samples for the next iteration. To the best of our
knowledge this is the first near-field framework which is able to accurately
predict 3D shape from highly specular objects. Our method outperforms competing
state-of-the-art near-field Photometric Stereo approaches on both synthetic and
real experiments.

该研究提出了一种 CNN 基于光度立体的方法，能够处理如光传播、透射视图几何和镜面光反射等现实假设，通过两个主要步骤迭代估计形状和辐射方向，取得比现有状态 - of-the-art 方法更好的实验效果。

基于 CNN 的近场光度立体问题方法

A CNN Based Approach for the Near-Field Photometric Stereo Problem

Most multi-view 3D reconstruction algorithms, especially when
shape-from-shading cues are used, assume that object appearance is
predominantly diffuse. To alleviate this restriction, we introduce S2Dnet, a
generative adversarial network for transferring multiple views of objects with
specular reflection into diffuse ones, so that multi-view reconstruction
methods can be applied more effectively. Our network extends unsupervised
image-to-image translation to multi-view "specular to diffuse" translation. To
preserve object appearance across multiple views, we introduce a Multi-View
Coherence loss (MVC) that evaluates the similarity and faithfulness of local
patches after the view-transformation. Our MVC loss ensures that the similarity
of local correspondences among multi-view images is preserved under the
image-to-image translation. As a result, our network yields significantly
better results than several single-view baseline techniques. In addition, we
carefully design and generate a large synthetic training data set using
physically-based rendering. During testing, our network takes only the raw
glossy images as input, without extra information such as segmentation masks or
lighting estimation. Results demonstrate that multi-view reconstruction can be
significantly improved using the images filtered by our network. We also show
promising performance on real world training and testing data.

该研究介绍了一种基于生成对抗网络的方法，将多视角高光反射的物体转换为漫反射形式，以更有效地应用于多视角三维重建，并通过 MVC 等措施保留了多视角图像间的局部对应关系，提高了成果的精度。