Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias. Although unsupervised image-to-image translation networks represented by CycleGAN show great potential in dealing with domain gap, it is non-trivial to generalize this method to stereo matching due to the problem of pixel distortion and stereo mismatch after translation. In this paper, we propose an end-to-end training framework with domain translation and stereo matching networks to tackle this challenge. First, joint optimization between domain translation and stereo matching networks in our end-to-end framework makes the former facilitate the latter one to the maximum extent. Second, this framework introduces two novel losses, i.e., bidirectional multi-scale feature re-projection loss and correlation consistency loss, to help translate all synthetic stereo images into realistic ones as well as maintain epipolar constraints. The effective combination of above two contributions leads to impressive stereo-consistent translation and disparity estimation accuracy. In addition, a mode seeking regularization term is added to endow the synthetic-to-real translation results with higher fine-grained diversity. Extensive experiments demonstrate the effectiveness of the proposed framework on bridging the synthetic-to-real domain gap on stereo matching.

本研究提出了一种端到端的域翻译和立体匹配网络训练框架，通过引入双向多尺度特征重投影损失和相关性一致性损失等两个新颖的损失函数来提高合成立体图像到真实图像的转换效果以及维护单应约束，实验证明进行域翻译和立体匹配网络联合优化能够有效地解决合成数据集所引入的领域偏差问题，从而实现更准确的立体匹配。

StereoGAN: 聚合域翻译和立体匹配的联合优化来桥接从合成到真实领域的差距