Given a set of calibrated images of a scene, we present an approach that
produces a simple, compact, and actionable 3D world representation by means of
3D primitives. While many approaches focus on recovering high-fidelity 3D
scenes, we focus on parsing a scene into mid-level 3D representations made of a
small set of textured primitives. Such representations are interpretable, easy
to manipulate and suited for physics-based simulations. Moreover, unlike
existing primitive decomposition methods that rely on 3D input data, our
approach operates directly on images through differentiable rendering.
Specifically, we model primitives as textured superquadric meshes and optimize
their parameters from scratch with an image rendering loss. We highlight the
importance of modeling transparency for each primitive, which is critical for
optimization and also enables handling varying numbers of primitives. We show
that the resulting textured primitives faithfully reconstruct the input images
and accurately model the visible 3D points, while providing amodal shape
completions of unseen object regions. We compare our approach to the state of
the art on diverse scenes from DTU, and demonstrate its robustness on real-life
captures from BlendedMVS and Nerfstudio. We also showcase how our results can
be used to effortlessly edit a scene or perform physical simulations. Code and
video results are available at this https URL .

本研究提出一种基于图像的方法，通过对纹理强的 3D 原语进行优化，得到了简单、紧凑且具有操作性的 3D 世界表示，并比较了该方法与现有方法在真实场景中的表现。