Neural Radiance Fields (NeRF) have demonstrated effectiveness in synthesizing
novel views. However, their reliance on dense inputs and scene-specific
optimization has limited their broader applicability. Generalizable NeRFs
(Gen-NeRF), while intended to address this, often produce blurring artifacts in
unobserved regions with sparse inputs, which are full of uncertainty. In this
paper, we aim to diminish the uncertainty in Gen-NeRF for plausible renderings.
We assume that NeRF's inability to effectively mitigate this uncertainty stems
from its inherent lack of generative capacity. Therefore, we innovatively
propose an Indirect Diffusion-guided NeRF framework, termed ID-NeRF, to address
this uncertainty from a generative perspective by leveraging a distilled
diffusion prior as guidance. Specifically, to avoid model confusion caused by
directly regularizing with inconsistent samplings as in previous methods, our
approach introduces a strategy to indirectly inject the inherently missing
imagination into the learned implicit function through a diffusion-guided
latent space. Empirical evaluation across various benchmarks demonstrates the
superior performance of our approach in handling uncertainty with sparse
inputs.

利用间接扩散引导的 NeRF (ID-NeRF) 框架通过利用精炼的扩散先验指导从生成性的角度解决 Gen-NeRF 中的不确定性，避免了直接约束的混乱取样导致的模型混淆，并经过多个基准测试表明在处理稀疏输入的不确定性方面具有卓越的性能。

通过间接扩散指导驯服稀疏视图通用 NeRF 中的不确定性

Taming Uncertainty in Sparse-view Generalizable NeRF via Indirect  Diffusion Guidance

Novel view synthesis is an essential functionality for enabling immersive
experiences in various Augmented- and Virtual-Reality (AR/VR) applications, for
which generalizable Neural Radiance Fields (NeRFs) have gained increasing
popularity thanks to their cross-scene generalization capability. Despite their
promise, the real-device deployment of generalizable NeRFs is bottlenecked by
their prohibitive complexity due to the required massive memory accesses to
acquire scene features, causing their ray marching process to be
memory-bounded. To this end, we propose Gen-NeRF, an algorithm-hardware
co-design framework dedicated to generalizable NeRF acceleration, which for the
first time enables real-time generalizable NeRFs. On the algorithm side,
Gen-NeRF integrates a coarse-then-focus sampling strategy, leveraging the fact
that different regions of a 3D scene contribute differently to the rendered
pixel, to enable sparse yet effective sampling. On the hardware side, Gen-NeRF
highlights an accelerator micro-architecture to maximize the data reuse
opportunities among different rays by making use of their epipolar geometric
relationship. Furthermore, our Gen-NeRF accelerator features a customized
dataflow to enhance data locality during point-to-hardware mapping and an
optimized scene feature storage strategy to minimize memory bank conflicts.
Extensive experiments validate the effectiveness of our proposed Gen-NeRF
framework in enabling real-time and generalizable novel view synthesis.

Gen-NeRF 是一个算法硬件协同设计框架，可以加速可广义的 NeRF (神经辐射场) 生成新视点，从而实现实时的虚拟和增强现实体验。