Disparity estimation for binocular stereo images finds a wide range of
applications. Traditional algorithms may fail on featureless regions, which
could be handled by high-level clues such as semantic segments. In this paper,
we suggest that appropriate incorporation of semantic cues can greatly rectify
prediction in commonly-used disparity estimation frameworks. Our method
conducts semantic feature embedding and regularizes semantic cues as the loss
term to improve learning disparity. Our unified model SegStereo employs
semantic features from segmentation and introduces semantic softmax loss, which
helps improve the prediction accuracy of disparity maps. The semantic cues work
well in both unsupervised and supervised manners. SegStereo achieves
state-of-the-art results on KITTI Stereo benchmark and produces decent
prediction on both CityScapes and FlyingThings3D datasets.

本研究提出了一种利用语义线索修正传统视差计算算法预测的方法，通过语义特征嵌入和语义 loss 正则化优化来提高学习视差的效果，并在 SegStereo 模型中整合了自分割中获取的语义特征和引入语义 softmax loss 以提高视差图像预测准确性，同时在无监督和有监督训练方式下都取得了业界领先的 KITTI 立体视觉 benchmark 和较为优秀的 CityScapes、FlyingThings3D 数据集上的预测结果。