Amodal object segmentation is a challenging task that involves segmenting
both visible and occluded parts of an object. In this paper, we propose a novel
approach, called Coarse-to-Fine Segmentation (C2F-Seg), that addresses this
problem by progressively modeling the amodal segmentation. C2F-Seg initially
reduces the learning space from the pixel-level image space to the
vector-quantized latent space. This enables us to better handle long-range
dependencies and learn a coarse-grained amodal segment from visual features and
visible segments. However, this latent space lacks detailed information about
the object, which makes it difficult to provide a precise segmentation
directly. To address this issue, we propose a convolution refine module to
inject fine-grained information and provide a more precise amodal object
segmentation based on visual features and coarse-predicted segmentation. To
help the studies of amodal object segmentation, we create a synthetic amodal
dataset, named as MOViD-Amodal (MOViD-A), which can be used for both image and
video amodal object segmentation. We extensively evaluate our model on two
benchmark datasets: KINS and COCO-A. Our empirical results demonstrate the
superiority of C2F-Seg. Moreover, we exhibit the potential of our approach for
video amodal object segmentation tasks on FISHBOWL and our proposed MOViD-A.
Project page at: this http URL

通过逐步建模视觉特征和粗略预测分割的精细信息，我们提出了一种名为 “Coarse-to-Fine Segmentation (C2F-Seg)” 的新方法来解决涉及对象可见和遮挡部分的不完全分割的挑战。同时，我们创建了一个合成的整体性数据集 MOViD-A，用于图像和视频整体性对象分割。通过在两个基准数据集 KINS 和 COCO-A 上进行大量评估，我们实证了 C2F-Seg 的优越性，并展示了在 FISHBOWL 和我们提出的 MOViD-A 上进行视频整体性对象分割任务的潜力。

基于形状先验的粗细阶段分离

Coarse-to-Fine Amodal Segmentation with Shape Prior

Passive visual systems typically fail to recognize objects in the amodal
setting where they are heavily occluded. In contrast, humans and other embodied
agents have the ability to move in the environment, and actively control the
viewing angle to better understand object shapes and semantics. In this work,
we introduce the task of Embodied Visual Recognition (EVR): An agent is
instantiated in a 3D environment close to an occluded target object, and is
free to move in the environment to perform object classification, amodal object
localization, and amodal object segmentation. To address this, we develop a new
model called Embodied Mask R-CNN, for agents to learn to move strategically to
improve their visual recognition abilities. We conduct experiments using the
House3D environment. Experimental results show that: 1) agents with embodiment
(movement) achieve better visual recognition performance than passive ones; 2)
in order to improve visual recognition abilities, agents can learn strategical
moving paths that are different from shortest paths.

本文提出了一种名为 “Embodied Visual Recognition” 的任务，其中代理可以在三维环境中移动以对被遮挡的目标物体进行识别、定位和分割，研究结果表明，具有移动能力的代理比被动代理的视觉识别性能更好，代理可以学习不同于最短路径的策略移动路径来提高视觉识别能力。