Semantic segmentation is a challenging problem due to difficulties in
modeling context in complex scenes and class confusions along boundaries. Most
literature either focuses on context modeling or boundary refinement, which is
less generalizable in open-world scenarios. In this work, we advocate a unified
framework(UN-EPT) to segment objects by considering both context information
and boundary artifacts. We first adapt a sparse sampling strategy to
incorporate the transformer-based attention mechanism for efficient context
modeling. In addition, a separate spatial branch is introduced to capture image
details for boundary refinement. The whole model can be trained in an
end-to-end manner. We demonstrate promising performance on three popular
benchmarks for semantic segmentation with low memory footprint. Code will be
released soon.

本文提出了一种统一的框架（UN-EPT）来解决复杂场景中的语义分割问题，该框架综合考虑了上下文信息和边界伪影。该模型采用了稀疏采样策略来提高上下文建模的效率，并引入了单独的空间分支来捕捉图像细节，整个模型可以端到端地训练。该方法在三个常用的语义分割数据集上表现出优秀的性能，并具有较低的内存占用。