Our method studies the complex task of object-centric 3D understanding from a single RGB-D observation. As it is an ill-posed problem, existing methods suffer from low performance for both 3D shape and 6D pose and size estimation in complex multi-object scenarios with occlusions. We present ShAPO, a method for joint multi-object detection, 3D textured reconstruction, 6D object pose and size estimation. Key to ShAPO is a single-shot pipeline to regress shape, appearance and pose latent codes along with the masks of each object instance, which is then further refined in a sparse-to-dense fashion. A novel disentangled shape and appearance database of priors is first learned to embed objects in their respective shape and appearance space. We also propose a novel, octree-based differentiable optimization step, allowing us to further improve object shape, pose and appearance simultaneously under the learned latent space, in an analysis-by-synthesis fashion. Our novel joint implicit textured object representation allows us to accurately identify and reconstruct novel unseen objects without having access to their 3D meshes. Through extensive experiments, we show that our method, trained on simulated indoor scenes, accurately regresses the shape, appearance and pose of novel objects in the real-world with minimal fine-tuning. Our method significantly out-performs all baselines on the NOCS dataset with an 8% absolute improvement in mAP for 6D pose estimation. Project page: https://zubair-irshad.github.io/projects/ShAPO.html

该研究提出了一种用于目标识别、3D萃取、6D姿态和大小估计的多任务检测流水线——ShAPO，其利用单激光雷达管道、分别处理形状、外观和姿态潜在编码以及各个实例的掩码进行多目标检测。在学习对象的形状和外观空间时，还建议了一种新型的解缠绕形状和外观先验库。同时，它还提出了一种新颖的、基于八叉树的可微优化步骤，使我们能够同时改善对象的形状、姿态和外观。最后，基于实验，我们演示了我们的方法在真实场景中的准确性，并证明了该方法的效果要明显优于现有方法，在6D姿态估计方面提高了8%的mAP。

ShAPO: 多目标形状、外观和姿态优化的隐式表征