Stereo matching and semantic segmentation are significant tasks in binocular
satellite 3D reconstruction. However, previous studies primarily view these as
independent parallel tasks, lacking an integrated multitask learning framework.
This work introduces a solution, the Single-branch Semantic Stereo Network
(S3Net), which innovatively combines semantic segmentation and stereo matching
using Self-Fuse and Mutual-Fuse modules. Unlike preceding methods that utilize
semantic or disparity information independently, our method dentifies and
leverages the intrinsic link between these two tasks, leading to a more
accurate understanding of semantic information and disparity estimation.
Comparative testing on the US3D dataset proves the effectiveness of our S3Net.
Our model improves the mIoU in semantic segmentation from 61.38 to 67.39, and
reduces the D1-Error and average endpoint error (EPE) in disparity estimation
from 10.051 to 9.579 and 1.439 to 1.403 respectively, surpassing existing
competitive methods. Our codes are available at:this https URL

提出了一种名为 Single-branch Semantic Stereo Network（S3Net）的解决方案，通过自我融合和互相融合模块创新性地将语义分割和立体匹配相结合，从而更准确地理解语义信息和视差估计。在 US3D 数据集上的比较测试证实了 S3Net 的有效性，相比现有竞争方法，我们的模型将语义分割的平均交并比（mIoU）从 61.38 提升至 67.39，将立体匹配的 D1 误差和平均端点误差（EPE）从 10.051 降低至 9.579 和从 1.439 降低至 1.403。