Recent researches on panoptic segmentation resort to a single end-to-end
network to combine the tasks of instance segmentation and semantic
segmentation. However, prior models only unified the two related tasks at the
architectural level via a multi-branch scheme or revealed the underlying
correlation between them by unidirectional feature fusion, which disregards the
explicit semantic and co-occurrence relations among objects and background.
Inspired by the fact that context information is critical to recognize and
localize the objects, and inclusive object details are significant to parse the
background scene, we thus investigate on explicitly modeling the correlations
between object and background to achieve a holistic understanding of an image
in the panoptic segmentation task. We introduce a Bidirectional Graph Reasoning
Network (BGRNet), which incorporates graph structure into the conventional
panoptic segmentation network to mine the intra-modular and intermodular
relations within and between foreground things and background stuff classes. In
particular, BGRNet first constructs image-specific graphs in both instance and
semantic segmentation branches that enable flexible reasoning at the proposal
level and class level, respectively. To establish the correlations between
separate branches and fully leverage the complementary relations between things
and stuff, we propose a Bidirectional Graph Connection Module to diffuse
information across branches in a learnable fashion. Experimental results
demonstrate the superiority of our BGRNet that achieves the new
state-of-the-art performance on challenging COCO and ADE20K panoptic
segmentation benchmarks.

通过在传统的全景分割网络中引入图形结构来挖掘前景物体和背景物体类别内的模块内部和模块间关系，进而建立单一的 end-to-end 网络，以实现对图像的整体理解。该模型 (BGRNet) 在 COCO 和 ADE20K 全景分割基准中实现了新的最先进性能。

双向图推理网络用于全景分割

Bidirectional Graph Reasoning Network for Panoptic Segmentation

Crowd counting is an important yet challenging task due to the large scale
and density variation. Recent investigations have shown that distilling rich
relations among multi-scale features and exploiting useful information from the
auxiliary task, i.e., localization, are vital for this task. Nevertheless, how
to comprehensively leverage these relations within a unified network
architecture is still a challenging problem. In this paper, we present a novel
network structure called Hybrid Graph Neural Network (HyGnn) which targets to
relieve the problem by interweaving the multi-scale features for crowd density
as well as its auxiliary task (localization) together and performing joint
reasoning over a graph. Specifically, HyGnn integrates a hybrid graph to
jointly represent the task-specific feature maps of different scales as nodes,
and two types of relations as edges:(i) multi-scale relations for capturing the
feature dependencies across scales and (ii) mutual beneficial relations
building bridges for the cooperation between counting and localization. Thus,
through message passing, HyGnn can distill rich relations between the nodes to
obtain more powerful representations, leading to robust and accurate results.
Our HyGnn performs significantly well on four challenging datasets:
ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF_QNRF, outperforming
the state-of-the-art approaches by a large margin.

本文提出了一种新的网络结构 —— 混合图神经网络（HyGnn），通过交织多尺度特征以及辅助任务（定位）来执行图推理，从而缓解了人群密度估计中的挑战问题，实现了比现有方法更为准确的效果。