A diffusion model learns to predict a vector field of gradients. We propose
to apply chain rule on the learned gradients, and back-propagate the score of a
diffusion model through the Jacobian of a differentiable renderer, which we
instantiate to be a voxel radiance field. This setup aggregates 2D scores at
multiple camera viewpoints into a 3D score, and repurposes a pretrained 2D
model for 3D data generation. We identify a technical challenge of distribution
mismatch that arises in this application, and propose a novel estimation
mechanism to resolve it. We run our algorithm on several off-the-shelf
diffusion image generative models, including the recently released Stable
Diffusion trained on the large-scale LAION dataset.

本文提出一种基于扩散模型的新方法，利用可微分渲染器的雅各比矩阵，将二维数据转化为三维数据生成，解决了分布不匹配的技术挑战，并将该算法应用于多个离线扩散图像生成模型。

Score Jacobian Chaining：将预训练的 2D 扩散模型提升至 3D 生成

Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D  Generation

Although various 3D datasets with different functions and scales have been
proposed recently, it remains challenging for individuals to complete the whole
pipeline of large-scale data collection, sanitization, and annotation.
Moreover, the created datasets usually suffer from extremely imbalanced class
distribution or partial low-quality data samples. Motivated by this, we explore
the procedurally synthetic 3D data generation paradigm to equip individuals
with the full capability of creating large-scale annotated photogrammetry point
clouds. Specifically, we introduce a synthetic aerial photogrammetry point
clouds generation pipeline that takes full advantage of open geospatial data
sources and off-the-shelf commercial packages. Unlike generating synthetic data
in virtual games, where the simulated data usually have limited gaming
environments created by artists, the proposed pipeline simulates the
reconstruction process of the real environment by following the same UAV flight
pattern on different synthetic terrain shapes and building densities, which
ensure similar quality, noise pattern, and diversity with real data. In
addition, the precise semantic and instance annotations can be generated fully
automatically, avoiding the expensive and time-consuming manual annotation.
Based on the proposed pipeline, we present a richly-annotated synthetic 3D
aerial photogrammetry point cloud dataset, termed STPLS3D, with more than 16
$km^2$ of landscapes and up to 18 fine-grained semantic categories. For
verification purposes, we also provide a parallel dataset collected from four
areas in the real environment. Extensive experiments conducted on our datasets
demonstrate the effectiveness and quality of the proposed synthetic dataset.

探索通过改进人们的能力来创造大规模注释的航空摄影测量点云的 3D 数据生成方法，利用开源地理空间数据和商业软件来生成精确的语义和实例注释，提出 STPLS3D 数据集并在其上进行的实验表明了该数据集的有效性和质量。

STPLS3D：一个大规模的合成和真实航空摄影测量三维点云数据集

STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point  Cloud Dataset

Generation of 3D data by deep neural network has been attracting increasing
attention in the research community. The majority of extant works resort to
regular representations such as volumetric grids or collection of images;
however, these representations obscure the natural invariance of 3D shapes
under geometric transformations and also suffer from a number of other issues.
In this paper we address the problem of 3D reconstruction from a single image,
generating a straight-forward form of output -- point cloud coordinates. Along
with this problem arises a unique and interesting issue, that the groundtruth
shape for an input image may be ambiguous. Driven by this unorthodox output
form and the inherent ambiguity in groundtruth, we design architecture, loss
function and learning paradigm that are novel and effective. Our final solution
is a conditional shape sampler, capable of predicting multiple plausible 3D
point clouds from an input image. In experiments not only can our system
outperform state-of-the-art methods on single image based 3d reconstruction
benchmarks; but it also shows a strong performance for 3d shape completion and
promising ability in making multiple plausible predictions.

使用深度神经网络从单一图像中重建 3D 点云坐标，设计了面对真实世界几何转换不变性和地面真实性模糊的问题的新型方法，包括条件形状采样器，能够预测多个可能的 3D 点云。在实验中表现优异，不仅在单图像 based 3D 重建基准测试中胜过现有技术，也在形状补全方面表现出强大性能，有望在多个可能性预测方面表现出色。