Panoptic segmentation unifies semantic and instance segmentation and thus
delivers a semantic class label and, for so-called thing classes, also an
instance label per pixel. The differentiation of distinct objects of the same
class with a similar appearance is particularly challenging and frequently
causes such objects to be incorrectly assigned to a single instance. In the
present work, we demonstrate that information on the 3D geometry of the
observed scene can be used to mitigate this issue: We present a novel CNN-based
method for panoptic segmentation which processes RGB images and depth maps
given as input in separate network branches and fuses the resulting feature
maps in a late fusion manner. Moreover, we propose a new depth-aware dice loss
term which penalises the assignment of pixels to the same thing instance based
on the difference between their associated distances to the camera. Experiments
carried out on the Cityscapes dataset show that the proposed method reduces the
number of objects that are erroneously merged into one thing instance and
outperforms the method used as basis by 2.2% in terms of panoptic quality.

基于 RGB 图像和深度图像的全景分割的新方法；通过使用所观察场景的 3D 几何信息，减少错误合并为一个事物实例的对象数量，提高全景质量。