We present a method that learns neural shadow fields which are neural scene
representations that are only learnt from the shadows present in the scene.
While traditional shape-from-shadow (SfS) algorithms reconstruct geometry from
shadows, they assume a fixed scanning setup and fail to generalize to complex
scenes. Neural rendering algorithms, on the other hand, rely on photometric
consistency between RGB images, but largely ignore physical cues such as
shadows, which have been shown to provide valuable information about the scene.
We observe that shadows are a powerful cue that can constrain neural scene
representations to learn SfS, and even outperform NeRF to reconstruct otherwise
hidden geometry. We propose a graphics-inspired differentiable approach to
render accurate shadows with volumetric rendering, predicting a shadow map that
can be compared to the ground truth shadow. Even with just binary shadow maps,
we show that neural rendering can localize the object and estimate coarse
geometry. Our approach reveals that sparse cues in images can be used to
estimate geometry using differentiable volumetric rendering. Moreover, our
framework is highly generalizable and can work alongside existing 3D
reconstruction techniques that otherwise only use photometric consistency.

该研究提出一种基于神经投影场的方法，利用阴影来学习场景的神经表示，实现了用稀疏图像信号估计几何信息的目的，并且证明了该方法的泛化能力，并且在与 NeRF 等其他方法的比较中体现出更好的性能。