We present a differentiable rendering framework to learn structured 3D
abstractions in the form of primitive assemblies from sparse RGB images
capturing a 3D object. By leveraging differentiable volume rendering, our
method does not require 3D supervision. Architecturally, our network follows
the general pipeline of an image-conditioned neural radiance field (NeRF)
exemplified by pixelNeRF for color prediction. As our core contribution, we
introduce differential primitive assembly (DPA) into NeRF to output a 3D
occupancy field in place of density prediction, where the predicted occupancies
serve as opacity values for volume rendering. Our network, coined DPA-Net,
produces a union of convexes, each as an intersection of convex quadric
primitives, to approximate the target 3D object, subject to an abstraction loss
and a masking loss, both defined in the image space upon volume rendering. With
test-time adaptation and additional sampling and loss designs aimed at
improving the accuracy and compactness of the obtained assemblies, our method
demonstrates superior performance over state-of-the-art alternatives for 3D
primitive abstraction from sparse views.

通过可微分的渲染框架，在不需要 3D 监督的情况下，从稀疏的 RGB 图像中学习结构化的三维抽象，以原始组件的形式捕获三维物体，并且通过引入差异化的原始装配，输出三维占据场作为密度预测的替代，以提高精度和紧凑性。

DPA-Net: 通过可区分的原始装配从稀疏视角实现的结构化 3D 抽象

DPA-Net: Structured 3D Abstraction from Sparse Views via Differentiable  Primitive Assembly

We introduce CAPRI-Net, a neural network for learning compact and
interpretable implicit representations of 3D computer-aided design (CAD)
models, in the form of adaptive primitive assemblies. Our network takes an
input 3D shape that can be provided as a point cloud or voxel grids, and
reconstructs it by a compact assembly of quadric surface primitives via
constructive solid geometry (CSG) operations. The network is self-supervised
with a reconstruction loss, leading to faithful 3D reconstructions with sharp
edges and plausible CSG trees, without any ground-truth shape assemblies. While
the parametric nature of CAD models does make them more predictable locally, at
the shape level, there is a great deal of structural and topological
variations, which present a significant generalizability challenge to
state-of-the-art neural models for 3D shapes. Our network addresses this
challenge by adaptive training with respect to each test shape, with which we
fine-tune the network that was pre-trained on a model collection. We evaluate
our learning framework on both ShapeNet and ABC, the largest and most diverse
CAD dataset to date, in terms of reconstruction quality, shape edges,
compactness, and interpretability, to demonstrate superiority over current
alternatives suitable for neural CAD reconstruction.

CAPRI-Net 是一种神经网络，可以通过自适应基元装配体的形式来学习三维计算机辅助设计模型的紧凑且可解释的隐式表示，可对提供为点云或体素网格的输入 3D 形状进行重构并通过构造实体几何 (CSG) 运算来获得齐次组合，该网络通过重建损失进行自监督，而不需要任何真实的形状装配。