In the realm of autonomous vehicle (AV) perception, comprehending 3D scenes
is paramount for tasks such as planning and mapping. Semantic scene completion
(SSC) aims to infer scene geometry and semantics from limited observations.
While camera-based SSC has gained popularity due to affordability and rich
visual cues, existing methods often neglect the inherent uncertainty in models.
To address this, we propose an uncertainty-aware camera-based 3D semantic scene
completion method ($\alpha$-SSC). Our approach includes an uncertainty
propagation framework from depth models (Depth-UP) to enhance geometry
completion (up to 11.58% improvement) and semantic segmentation (up to 14.61%
improvement). Additionally, we propose a hierarchical conformal prediction
(HCP) method to quantify SSC uncertainty, effectively addressing high-level
class imbalance in SSC datasets. On the geometry level, we present a novel KL
divergence-based score function that significantly improves the occupied recall
of safety-critical classes (45% improvement) with minimal performance overhead
(3.4% reduction). For uncertainty quantification, we demonstrate the ability to
achieve smaller prediction set sizes while maintaining a defined coverage
guarantee. Compared with baselines, it achieves up to 85% reduction in set
sizes. Our contributions collectively signify significant advancements in SSC
accuracy and robustness, marking a noteworthy step forward in autonomous
perception systems.

提出了一种考虑不确定性的基于摄像头的三维语义场景完成方法，同时提出了一种层次化的置信度预测方法，能够增强几何完成和语义分割准确性，提高安全关键类别的占用召回率，实现更小的预测集大小并保持指定的覆盖保证，为自主感知系统的准确性和鲁棒性带来显著进展。

$α$-SSC：基于摄像头的不确定性感知的 3D 语义场景补全

$α$-SSC: Uncertainty-Aware Camera-based 3D Semantic Scene  Completion

Despite the emergence of successful NeRF inpainting methods built upon
explicit RGB and depth 2D inpainting supervisions, these methods are inherently
constrained by the capabilities of their underlying 2D inpainters. This is due
to two key reasons: (i) independently inpainting constituent images results in
view-inconsistent imagery, and (ii) 2D inpainters struggle to ensure
high-quality geometry completion and alignment with inpainted RGB images.
To overcome these limitations, we propose a novel approach called MVIP-NeRF
that harnesses the potential of diffusion priors for NeRF inpainting,
addressing both appearance and geometry aspects. MVIP-NeRF performs joint
inpainting across multiple views to reach a consistent solution, which is
achieved via an iterative optimization process based on Score Distillation
Sampling (SDS). Apart from recovering the rendered RGB images, we also extract
normal maps as a geometric representation and define a normal SDS loss that
motivates accurate geometry inpainting and alignment with the appearance.
Additionally, we formulate a multi-view SDS score function to distill
generative priors simultaneously from different view images, ensuring
consistent visual completion when dealing with large view variations. Our
experimental results show better appearance and geometry recovery than previous
NeRF inpainting methods.

提出了一种名为 MVIP-NeRF 的新方法，通过使用扩散先验来进行 NeRF 修复，解决了外观和几何方面的问题，实现了多视图共同修复和对齐。实验证明，与先前的 NeRF 修复方法相比，该方法在外观和几何恢复方面效果更好。

MVIP-NeRF：通过扩散先验进行 NeRF 场景的多视图 3D 修复

MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior

Prior work to infer 3D texture use either texture atlases, which require
uv-mappings and hence have discontinuities, or colored voxels, which are memory
inefficient and limited in resolution. Recent work, predicts RGB color at every
XYZ coordinate forming a texture field, but focus on completing texture given a
single 2D image. Instead, we focus on 3D texture and geometry completion from
partial and incomplete 3D scans. IF-Nets have recently achieved
state-of-the-art results on 3D geometry completion using a multi-scale deep
feature encoding, but the outputs lack texture. In this work, we generalize
IF-Nets to texture completion from partial textured scans of humans and
arbitrary objects. Our key insight is that 3D texture completion benefits from
incorporating local and global deep features extracted from both the 3D partial
texture and completed geometry. Specifically, given the partial 3D texture and
the 3D geometry completed with IF-Nets, our model successfully in-paints the
missing texture parts in consistence with the completed geometry. Our model won
the SHARP ECCV'20 challenge, achieving highest performance on all challenges.

本研究旨在实现通过 IF-Nets 模型进行基于局部和全局深度特征提取，从部分三维纹理和已完成几何体上完成 3D 纹理的完整。我们的模型成功地用靠近已完成几何体的内插法来处理丢失的纹理部分，并在 SHARP ECCV20 挑战中取得了最高表现。