Modern semantic segmentation methods devote much attention to adjusting feature representations to improve the segmentation performance in various ways, such as metric learning, architecture design, etc. However, almost all those methods neglect the particularity of boundary pixels. These pixels are prone to obtain confusing features from both sides due to the continuous expansion of receptive fields in CNN networks. In this way, they will mislead the model optimization direction and make the class weights of such categories that tend to share many adjacent pixels lack discrimination, which will damage the overall performance. In this work, we dive deep into this problem and propose a novel method named Embedded Superpixel CRF (ES-CRF) to address it. ES-CRF involves two main aspects. On the one hand, ES-CRF innovatively fuses the CRF mechanism into the CNN network as an organic whole for more effective end-to-end optimization. It utilizes CRF to guide the message passing between pixels in high-level features to purify the feature representation of boundary pixels, with the help of inner pixels belong to the same object. On the other hand, superpixel is integrated into ES-CRF to exploit the local object prior for more reliable message passing. Finally, our proposed method yields new records on two challenging benchmarks, i.e., Cityscapes and ADE20K. Moreover, we make detailed theoretical analysis to verify the superiority of ES-CRF.

本文提出了一种名为Embedded Conditional Random Field (E-CRF)的方法，通过将CRF与CNN网络有机融合来更有效地进行端到端优化，从而解决分割模型中类别权重的问题，特别关注的是由于邻接边界像素共享而导致的类别权重混淆问题。方法中利用了CRF引导了高层特征之间的信息传递，利用相同对象内像素来净化边界像素的特征表示，并且在反向传播期间优化类别权重。实验结果在ADE20K、Cityscapes和Pascal Context数据集上表现出了令人瞩目的效果。

嵌入式条件随机场用于语义分割中边界引起的类别权重混淆