The low-level spatial detail information and high-level semantic abstract
information are both essential to the semantic segmentation task. The features
extracted by the deep network can obtain rich semantic information, while a lot
of spatial information is lost. However, how to recover spatial detail
information effectively and fuse it with high-level semantics has not been well
addressed so far. In this paper, we propose a new architecture based on
Bilateral Segmentation Network (BiseNet) called Multi-scale Covariance Feature
Fusion Network (MCFNet). Specifically, this network introduces a new feature
refinement module and a new feature fusion module. Furthermore, a gating unit
named L-Gate is proposed to filter out invalid information and fuse multi-scale
features. We evaluate our proposed model on Cityscapes, CamVid datasets and
compare it with the state-of-the-art methods. Extensive experiments show that
our method achieves competitive success. On Cityscapes, we achieve 75.5% mIOU
with a speed of 151.3 FPS.

基于 BiseNet 的多尺度协方差特征融合网络 (MCFNet) 提出了一种新的架构，引入了新的特征细化模块和特征融合模块，并提出了名为 L-Gate 的门控单元来过滤无效信息和融合多尺度特征，实验结果显示我们的方法在 Cityscapes 和 CamVid 数据集上取得了竞争性的成功，其中在 Cityscapes 数据集上获得了 75.5% 的平均交并比 (mIOU) 和 151.3FPS 的速度。

MCFNet: 实时语义分割的多尺度协方差特征融合网络

MCFNet: Multi-scale Covariance Feature Fusion Network for Real-time  Semantic Segmentation

BiSeNet has been proved to be a popular two-stream network for real-time
segmentation. However, its principle of adding an extra path to encode spatial
information is time-consuming, and the backbones borrowed from pretrained
tasks, e.g., image classification, may be inefficient for image segmentation
due to the deficiency of task-specific design. To handle these problems, we
propose a novel and efficient structure named Short-Term Dense Concatenate
network (STDC network) by removing structure redundancy. Specifically, we
gradually reduce the dimension of feature maps and use the aggregation of them
for image representation, which forms the basic module of STDC network. In the
decoder, we propose a Detail Aggregation module by integrating the learning of
spatial information into low-level layers in single-stream manner. Finally, the
low-level features and deep features are fused to predict the final
segmentation results. Extensive experiments on Cityscapes and CamVid dataset
demonstrate the effectiveness of our method by achieving promising trade-off
between segmentation accuracy and inference speed. On Cityscapes, we achieve
71.9% mIoU on the test set with a speed of 250.4 FPS on NVIDIA GTX 1080Ti,
which is 45.2% faster than the latest methods, and achieve 76.8% mIoU with 97.0
FPS while inferring on higher resolution images.

提出了一种高效的 STDC 网络和 Detail Aggregation 模块，用于实时图像分割，通过在低级别层中学习空间信息，并将低级和深层特征融合以预测最终的分割结果，取得了较高的分割准确率和快速推理速度。