We propose PyTorchGeoNodes, a differentiable module for reconstructing 3D
objects from images using interpretable shape programs. In comparison to
traditional CAD model retrieval methods, the use of shape programs for 3D
reconstruction allows for reasoning about the semantic properties of
reconstructed objects, editing, low memory footprint, etc. However, the
utilization of shape programs for 3D scene understanding has been largely
neglected in past works. As our main contribution, we enable gradient-based
optimization by introducing a module that translates shape programs designed in
Blender, for example, into efficient PyTorch code. We also provide a method
that relies on PyTorchGeoNodes and is inspired by Monte Carlo Tree Search
(MCTS) to jointly optimize discrete and continuous parameters of shape programs
and reconstruct 3D objects for input scenes. In our experiments, we apply our
algorithm to reconstruct 3D objects in the ScanNet dataset and evaluate our
results against CAD model retrieval-based reconstructions. Our experiments
indicate that our reconstructions match well the input scenes while enabling
semantic reasoning about reconstructed objects.

我们提出了 PyTorchGeoNodes，一种可解释的形状程序，用于从图像中重建 3D 对象。我们的实验表明，我们的重建结果与输入场景很好地匹配，并能对重建对象进行语义推理。