Current 3D reconstruction techniques struggle to infer unbounded scenes from
a few images faithfully. Specifically, existing methods have high computational
demands, require detailed pose information, and cannot reconstruct occluded
regions reliably. We introduce 6Img-to-3D, an efficient, scalable
transformer-based encoder-renderer method for single-shot image to 3D
reconstruction. Our method outputs a 3D-consistent parameterized triplane from
only six outward-facing input images for large-scale, unbounded outdoor driving
scenarios. We take a step towards resolving existing shortcomings by combining
contracted custom cross- and self-attention mechanisms for triplane
parameterization, differentiable volume rendering, scene contraction, and image
feature projection. We showcase that six surround-view vehicle images from a
single timestamp without global pose information are enough to reconstruct
360$^{\circ}$ scenes during inference time, taking 395 ms. Our method allows,
for example, rendering third-person images and birds-eye views. Our code is
available at this https URL, and more examples can
be found at our website here this https URL

通过结合压缩的自注意力机制、不可微分体素渲染、场景压缩和图像特征投影等方法，我们提出了一种高效、可扩展的基于 Transformer 的编码器 - 渲染器方法，用于从单次拍摄的图像重建 3D 一致的参数化三平面，适用于大规模、无界的户外驾驶场景。

6Img-to-3D: 少图像大规模室外行驶场景重建

6Img-to-3D: Few-Image Large-Scale Outdoor Driving Scene Reconstruction

Though neural radiance fields (NeRF) have demonstrated impressive view
synthesis results on objects and small bounded regions of space, they struggle
on "unbounded" scenes, where the camera may point in any direction and content
may exist at any distance. In this setting, existing NeRF-like models often
produce blurry or low-resolution renderings (due to the unbalanced detail and
scale of nearby and distant objects), are slow to train, and may exhibit
artifacts due to the inherent ambiguity of the task of reconstructing a large
scene from a small set of images. We present an extension of mip-NeRF (a NeRF
variant that addresses sampling and aliasing) that uses a non-linear scene
parameterization, online distillation, and a novel distortion-based regularizer
to overcome the challenges presented by unbounded scenes. Our model, which we
dub "mip-NeRF 360" as we target scenes in which the camera rotates 360 degrees
around a point, reduces mean-squared error by 57% compared to mip-NeRF, and is
able to produce realistic synthesized views and detailed depth maps for highly
intricate, unbounded real-world scenes.

本研究提出了一种名为 mĩp-NeRF 360 的模型，通过应用非线性场景参数化、在线蒸馏和一种新的基于畸变的正则化器，克服了 3D 场景中存在的挑战，在高度复杂的未限定真实世界场景中减少了均方误差 57％，能够提供逼真的合成视图和详细的深度图。