Panoramic imaging research on geometry recovery and High Dynamic Range (HDR)
reconstruction becomes a trend with the development of Extended Reality (XR).
Neural Radiance Fields (NeRF) provide a promising scene representation for both
tasks without requiring extensive prior data. However, in the case of inputting
sparse Low Dynamic Range (LDR) panoramic images, NeRF often degrades with
under-constrained geometry and is unable to reconstruct HDR radiance from LDR
inputs. We observe that the radiance from each pixel in panoramic images can be
modeled as both a signal to convey scene lighting information and a light
source to illuminate other pixels. Hence, we propose the irradiance fields from
sparse LDR panoramic images, which increases the observation counts for
faithful geometry recovery and leverages the irradiance-radiance attenuation
for HDR reconstruction. Extensive experiments demonstrate that the irradiance
fields outperform state-of-the-art methods on both geometry recovery and HDR
reconstruction and validate their effectiveness. Furthermore, we show a
promising byproduct of spatially-varying lighting estimation. The code is
available at this https URL

稀疏低动态范围（LDR）全景图像中辐照度场的模型提高了几何恢复和高动态范围（HDR）重建的能力，并且在实验证明了其有效性。

Pano-NeRF：从稀疏低动态范围全景图像合成具有几何图形的高动态范围新视点

Pano-NeRF: Synthesizing High Dynamic Range Novel Views with Geometry  from Sparse Low Dynamic Range Panoramic Images

We address the challenge of recovering an underlying scene geometry and
colors from a sparse set of RGBD view observations. In this work, we present a
new solution termed RGBD$^2$ that sequentially generates novel RGBD views along
a camera trajectory, and the scene geometry is simply the fusion result of
these views. More specifically, we maintain an intermediate surface mesh used
for rendering new RGBD views, which subsequently becomes complete by an
inpainting network; each rendered RGBD view is later back-projected as a
partial surface and is supplemented into the intermediate mesh. The use of
intermediate mesh and camera projection helps solve the tough problem of
multi-view inconsistency. We practically implement the RGBD inpainting network
as a versatile RGBD diffusion model, which is previously used for 2D generative
modeling; we make a modification to its reverse diffusion process to enable our
use. We evaluate our approach on the task of 3D scene synthesis from sparse
RGBD inputs; extensive experiments on the ScanNet dataset demonstrate the
superiority of our approach over existing ones. Project page:
this https URL

该研究提出了一个新的解决方案 - RGBD^2，它可以从稀疏的 RGBD 视图中顺序生成新的 RGBD 视图，并将场景几何形状作为这些视图的融合结果。通过维护中间表面网格，并使用摄像机投影，将渲染的 RGBD 视图再次投影为部分表面并补充到中间网格中，以解决多视角不一致性的问题。这种方法在稀疏的 RGBD 输入中合成 3D 场景时具有优越性。

RGBD2：使用 RGBD 扩散模型进行增量式视图修复的生成场景合成

RGBD2: Generative Scene Synthesis via Incremental View Inpainting using  RGBD Diffusion Models

While recovery of geometry from image and video data has received a lot of
attention in computer vision, methods to capture the texture for a given
geometry are less mature. Specifically, classical methods for texture
generation often assume clean geometry and reasonably well-aligned image data.
While very recent methods, e.g., adversarial texture optimization, better
handle lower-quality data obtained from hand-held devices, we find them to
still struggle frequently. To improve robustness, particularly of recent
adversarial texture optimization, we develop an explicit initialization and an
alignment procedure. It deals with complex geometry due to a robust mapping of
the geometry to the texture map and a hard-assignment-based initialization. It
deals with misalignment of geometry and images by integrating fast
image-alignment into the texture refinement optimization. We demonstrate
efficacy of our texture generation on a dataset of 11 scenes with a total of
2807 frames, observing 7.8% and 11.1% relative improvements regarding
perceptual and sharpness measurements.

本研究提出了一种基于几何图形的纹理生成方法，通过引入显式初始化和对齐程序来提高最新的对抗性纹理优化的鲁棒性，并将其应用于计算机视觉中的 11 个场景数据集中，能够获得相对于感知和锐度度量方面的 7.8％和 11.1％的相对改善。