We aim to generate fine-grained 3D geometry from large-scale sparse LiDAR
scans, abundantly captured by autonomous vehicles (AV). Contrary to prior work
on AV scene completion, we aim to extrapolate fine geometry from unlabeled and
beyond spatial limits of LiDAR scans, taking a step towards generating
realistic, high-resolution simulation-ready 3D street environments. We propose
hierarchical Generative Cellular Automata (hGCA), a spatially scalable
conditional 3D generative model, which grows geometry recursively with local
kernels following, in a coarse-to-fine manner, equipped with a light-weight
planner to induce global consistency. Experiments on synthetic scenes show that
hGCA generates plausible scene geometry with higher fidelity and completeness
compared to state-of-the-art baselines. Our model generalizes strongly from
sim-to-real, qualitatively outperforming baselines on the Waymo-open dataset.
We also show anecdotal evidence of the ability to create novel objects from
real-world geometric cues even when trained on limited synthetic content. More
results and details can be found on
this https URL

我们通过使用层级生成元胞自动机 (hGCA) 模型，从大规模稀疏的自动驾驶车辆 (AV) 捕捉到的 LiDAR 扫描数据中生成精细的三维几何结构，为模拟真实、高分辨率的街景环境迈出了一步。

层次生成细胞自动机在室外场景外推中的应用

Outdoor Scene Extrapolation with Hierarchical Generative Cellular  Automata

We propose a probabilistic shape completion method extended to the continuous
geometry of large-scale 3D scenes. Real-world scans of 3D scenes suffer from a
considerable amount of missing data cluttered with unsegmented objects. The
problem of shape completion is inherently ill-posed, and high-quality result
requires scalable solutions that consider multiple possible outcomes. We employ
the Generative Cellular Automata that learns the multi-modal distribution and
transform the formulation to process large-scale continuous geometry. The local
continuous shape is incrementally generated as a sparse voxel embedding, which
contains the latent code for each occupied cell. We formally derive that our
training objective for the sparse voxel embedding maximizes the variational
lower bound of the complete shape distribution and therefore our progressive
generation constitutes a valid generative model. Experiments show that our
model successfully generates diverse plausible scenes faithful to the input,
especially when the input suffers from a significant amount of missing data. We
also demonstrate that our approach outperforms deterministic models even in
less ambiguous cases with a small amount of missing data, which infers that
probabilistic formulation is crucial for high-quality geometry completion on
input scans exhibiting any levels of completeness.

该研究提出了一种面向大规模 3D 场景连续几何的概率形状完成方法，该方法利用生成细胞自动机学习多模态分布，通过稀疏体素嵌入渐进式生成连续形状，其训练目标最大化完整形状分布的变分下限，并通过实验证明了相比于确定性模型，该方法在几何完成方面的表现更佳。