Semantic Scene Completion (SSC) aims to jointly infer semantics and
occupancies of 3D scenes. Truncated Signed Distance Function (TSDF), a 3D
encoding of depth, has been a common input for SSC. Furthermore, RGB-TSDF
fusion, seems promising since these two modalities provide color and geometry
information, respectively. Nevertheless, RGB-TSDF fusion has been considered
nontrivial and commonly-used naive addition will result in inconsistent
results. We argue that the inconsistency comes from the sparsity of RGB
features upon projecting into 3D space, while TSDF features are dense, leading
to imbalanced feature maps when summed up. To address this RGB-TSDF
distribution difference, we propose a two-stage network with a 3D RGB feature
completion module that completes RGB features with meaningful values for
occluded areas. Moreover, we propose an effective classwise entropy loss
function to punish inconsistency. Extensive experiments on public datasets
verify that our method achieves state-of-the-art performance among methods that
do not adopt extra data.

基于 RGB-TSDF 融合的 3D 场景补全方法中存在局部密度差异的问题以及不一致性，本文提出了一个两阶段网络，包括一个 3D RGB 特征补全模块用来处理 RGB 特征在 3D 空间中的稀疏性问题，并且引入了一个有效的类内熵损失函数来惩罚不一致性，实验证明本文方法在不使用额外数据的情况下可以获得最先进的性能。

基于平衡的 RGB-TSDF 融合方法用于一致的语义场景补全：通过 3D RGB 特征补全与按类别分的熵损失函数

Towards Balanced RGB-TSDF Fusion for Consistent Semantic Scene  Completion by 3D RGB Feature Completion and a Classwise Entropy Loss Function

We present DiffRoom, a novel framework for tackling the problem of
high-quality 3D indoor room reconstruction and generation, both of which are
challenging due to the complexity and diversity of the room geometry. Although
diffusion-based generative models have previously demonstrated impressive
performance in image generation and object-level 3D generation, they have not
yet been applied to room-level 3D generation due to their computationally
intensive costs. In DiffRoom, we propose a sparse 3D diffusion network that is
efficient and possesses strong generative performance for Truncated Signed
Distance Field (TSDF), based on a rough occupancy prior. Inspired by
KinectFusion's incremental alignment and fusion of local SDFs, we propose a
diffusion-based TSDF fusion approach that iteratively diffuses and fuses TSDFs,
facilitating the reconstruction and generation of an entire room environment.
Additionally, to ease training, we introduce a curriculum diffusion learning
paradigm that speeds up the training convergence process and enables
high-quality reconstruction. According to the user study, the mesh quality
generated by our DiffRoom can even outperform the ground truth mesh provided by
ScanNet.

DiffRoom 提出了一种稀疏的 3D 扩散网络，它基于粗糙的占用先验和分层扩散学习结构，实现了高质量的 3D 室内重建和生成。

DiffRoom: 基于扩散的高质量三维房间重建与生成

DiffRoom: Diffusion-based High-Quality 3D Room Reconstruction and  Generation

Incrementally recovering 3D dense structures from monocular videos is of
paramount importance since it enables various robotics and AR applications.
Feature volumes have recently been shown to enable efficient and accurate
incremental dense reconstruction without the need to first estimate depth, but
they are not able to achieve as high of a resolution as depth-based methods due
to the large memory consumption of high-resolution feature volumes. This letter
proposes a real-time feature volume-based dense reconstruction method that
predicts TSDF (Truncated Signed Distance Function) values from a novel
sparsified deep feature volume, which is able to achieve higher resolutions
than previous feature volume-based methods, and is favorable in large-scale
outdoor scenarios where the majority of voxels are empty. An uncertainty-aware
multi-view stereo (MVS) network is leveraged to infer initial voxel locations
of the physical surface in a sparse feature volume. Then for refining the
recovered 3D geometry, deep features are attentively aggregated from multiview
images at potential surface locations, and temporally fused. Besides achieving
higher resolutions than before, our method is shown to produce more complete
reconstructions with finer detail in many cases. Extensive evaluations on both
public and self-collected datasets demonstrate a very competitive real-time
reconstruction result for our method compared to state-of-the-art
reconstruction methods in both indoor and outdoor settings.

本文提出了基于特征体积的实时密集重建方法，采用稀疏的深度特征体积来预测 TSDF 值，通过多视角图像聚合细节信息进行时序融合，实现了比以往更高分辨率和更完整细节的三维几何图形重建，其在室内和室外场景中都比现有的方法具有更好的实时重建性能。