Segmenting highly-overlapping image objects is challenging, because there is
typically no distinction between real object contours and occlusion boundaries
on images. Unlike previous instance segmentation methods, we model image
formation as a composition of two overlapping layers, and propose Bilayer
Convolutional Network (BCNet), where the top layer detects occluding objects
(occluders) and the bottom layer infers partially occluded instances
(occludees). The explicit modeling of occlusion relationship with bilayer
structure naturally decouples the boundaries of both the occluding and occluded
instances, and considers the interaction between them during mask regression.
We investigate the efficacy of bilayer structure using two popular
convolutional network designs, namely, Fully Convolutional Network (FCN) and
Graph Convolutional Network (GCN). Further, we formulate bilayer decoupling
using the vision transformer (ViT), by representing instances in the image as
separate learnable occluder and occludee queries. Large and consistent
improvements using one/two-stage and query-based object detectors with various
backbones and network layer choices validate the generalization ability of
bilayer decoupling, as shown by extensive experiments on image instance
segmentation benchmarks (COCO, KINS, COCOA) and video instance segmentation
benchmarks (YTVIS, OVIS, BDD100K MOTS), especially for heavy occlusion cases.
Code and data are available at this https URL

通过 Bilayer 卷积网络和视觉 Transformer 对遮挡的目标进行建模，以同时检测遮挡物和部分遮挡实例，并使用遮挡物和遮挡实例查询将遮挡关系分离，从而实现图像实例分割。

基于双层网络结构的遮挡感知实例分割

Occlusion-Aware Instance Segmentation via BiLayer Network Architectures

Most previous works of outdoor instance segmentation for images only use
color information. We explore a novel direction of sensor fusion to exploit
stereo cameras. Geometric information from disparities helps separate
overlapping objects of the same or different classes. Moreover, geometric
information penalizes region proposals with unlikely 3D shapes thus suppressing
false positive detections. Mask regression is based on 2D, 2.5D, and 3D ROI
using the pseudo-lidar and image-based representations. These mask predictions
are fused by a mask scoring process. However, public datasets only adopt stereo
systems with shorter baseline and focal legnth, which limit measuring ranges of
stereo cameras. We collect and utilize High-Quality Driving Stereo (HQDS)
dataset, using much longer baseline and focal length with higher resolution.
Our performance attains state of the art. Please refer to our project page. The
full paper is available here.

利用双目摄像机的传感器融合提高室外图像的对象分割，通过几何信息的利用，可以区分重叠的同类或异类物体，并抑制虚假检测，该方法的融合过程基于面罩评分，采用伪激光雷达和基于图像的表示进行 2D、2.5D 和 3D ROI 上的面罩回归。通过使用具有更高分辨率、更长基线和焦距的 High-Quality Driving Stereo（HQDS）数据集，我们的性能达到了最新的技术水平。