Deoccluding the hidden portions of objects in a scene is a formidable task,
particularly when addressing real-world scenes. In this paper, we present a new
self-supervised PArallel visible-to-COmplete diffusion framework, named PACO, a
foundation model for object-level scene deocclusion. Leveraging the rich prior
of pre-trained models, we first design the parallel variational autoencoder,
which produces a full-view feature map that simultaneously encodes multiple
complete objects, and the visible-to-complete latent generator, which learns to
implicitly predict the full-view feature map from partial-view feature map and
text prompts extracted from the incomplete objects in the input image. To train
PACO, we create a large-scale dataset with 500k samples to enable
self-supervised learning, avoiding tedious annotations of the amodal masks and
occluded regions. At inference, we devise a layer-wise deocclusion strategy to
improve efficiency while maintaining the deocclusion quality. Extensive
experiments on COCOA and various real-world scenes demonstrate the superior
capability of PACO for scene deocclusion, surpassing the state of the arts by a
large margin. Our method can also be extended to cross-domain scenes and novel
categories that are not covered by the training set. Further, we demonstrate
the deocclusion applicability of PACO in single-view 3D scene reconstruction
and object recomposition.

利用预训练模型的丰富先验知识，该论文提出了一种新的自监督并行可见完全扩散框架 (PACO)，用于对象级场景可视化处理，通过并行变分自动编码器和隐变量生成器，实现了在输入图像中从不完整对象的部分视图特征和文本提示隐含地预测完整对象的全视图特征，以实现 PACO 的自监督训练和层间遮挡策略的快速有效去遮挡，通过对 COCOA 数据集和多个现实场景的广泛实验，证明了 PACO 在场景去遮挡方面具有优越能力，同时超过了现有方法的性能，同时在单视图 3D 场景重建和物体重组方面具有广泛应用性。