Processing an input signal that contains arbitrary structures, e.g.,
superpixels and point clouds, remains a big challenge in computer vision.
Linear diffusion, an effective model for image processing, has been recently
integrated with deep learning algorithms. In this paper, we propose to learn
pairwise relations among data points in a global fashion to improve semantic
segmentation with arbitrarily-structured data, through spatial generalized
propagation networks (SGPN). The network propagates information on a group of
graphs, which represent the arbitrarily-structured data, through a learned,
linear diffusion process. The module is flexible to be embedded and jointly
trained with many types of networks, e.g., CNNs. We experiment with semantic
segmentation networks, where we use our propagation module to jointly train on
different data -- images, superpixels and point clouds. We show that SGPN
consistently improves the performance of both pixel and point cloud
segmentation, compared to networks that do not contain this module. Our method
suggests an effective way to model the global pairwise relations for
arbitrarily-structured data.

本研究提出了一种利用全局方式学习数据点之间的配对关系，通过空间广义传播网络（SGPN）实现对任意结构数据的语义分割的模型，该模块可以灵活地嵌入和联合训练许多类型的网络，如 CNN，并且实验证明，相较于不包含此模块的网络，SGPN 始终提高了像素和点云分割的性能，这表明我们的方法是模拟任意构造数据的全局配对关系的有效方法。