We propose the concept of a multi-frame GAN (MFGAN) and demonstrate its potential as an image sequence enhancement for stereo visual odometry in low light conditions. We base our method on an invertible adversarial network to transfer the beneficial features of brightly illuminated scenes to the sequence in poor illumination without costly paired datasets. In order to preserve the coherent geometric cues for the translated sequence, we present a novel network architecture as well as a novel loss term combining temporal and stereo consistencies based on optical flow estimation. We demonstrate that the enhanced sequences improve the performance of state-of-the-art feature-based and direct stereo visual odometry methods on both synthetic and real datasets in challenging illumination. We also show that MFGAN outperforms other state-of-the-art image enhancement and style transfer methods by a large margin in terms of visual odometry.

本文提出了一种基于多帧GAN的图像序列增强方法，并通过实验验证了其在低光条件下用于立体视觉里程计的潜力。该方法不需要昂贵的成对数据集，通过可逆对抗网络的优秀性能，将亮度较高的场景中的有益特征传输到低照度序列中，并通过引入一个基于光流估计的时间和立体一致性的新型损失函数，保留了翻译序列的连贯几何线索。实验表明，该增强序列在具有挑战性的光照条件下，提高了最先进的基于特征和基于直接立体视觉里程计方法的性能，并且相比其他最先进的图像增强和风格转移方法，MFGAN在视觉里程计方面表现出明显优势。

多帧 GAN：低光环境下用于立体视觉里程计图像增强