Recently, referring image segmentation has aroused widespread interest. Previous methods perform the multi-modal fusion between language and vision at the decoding side of the network. And, linguistic feature interacts with visual feature of each scale separately, which ignores the continuous guidance of language to multi-scale visual features. In this work, we propose an encoder fusion network (EFN), which transforms the visual encoder into a multi-modal feature learning network, and uses language to refine the multi-modal features progressively. Moreover, a co-attention mechanism is embedded in the EFN to realize the parallel update of multi-modal features, which can promote the consistent of the cross-modal information representation in the semantic space. Finally, we propose a boundary enhancement module (BEM) to make the network pay more attention to the fine structure. The experiment results on four benchmark datasets demonstrate that the proposed approach achieves the state-of-the-art performance under different evaluation metrics without any post-processing.

本论文提出了一种图像分割方法，即采用编码器融合网络（EFN）将视觉编码器转换为多模态特征学习网络，利用语言逐步优化多模态特征；在 EFN 中嵌入了协同注意机制，使得多模态特征的平行更新更具协同性，同时提出了边界增强模块（BEM）来增强网络对细节的关注，在四个基准数据集上的实验结果表明所提出的方法在不同的评价指标下均取得了最先进的性能，且无需后处理。

编码器融合网络和协同关注嵌入用于参照图像分割