CNN-based optical flow estimators have attracted attentions recently, mainly due to their impressive speed. As successful as they've been on synthetic datasets, they are still far behind the classical methods in real-world scenarios, mainly due to lack of flow ground-truth. In the current work, we seek to boost CNN-based flow estimation in real scenes with the help of the freely available self-supervised task of next-frame prediction. To this end we train the network in a hybrid way, providing it with a mixture of synthetic and real videos. With the help of a novel time-variant multi-tasking architecture, the network decides which of the tasks to learn at each point of time, depending on the availability of ground-truth. Our proposed method improves optical flow estimation in real scenes dramatically. We also experiment with prediction of "next-flow" instead of estimation of the current flow, which is intuitively more related to the task of next-frame prediction and improve the results even more. We report and evaluate results both qualitatively and quantitatively. The latter is done by training a single-stream action classifier on the estimated flow fields on UCF101 & HMDB51 and demonstrate high improvements of accuracy over the baseline: 10.2% and 9.6% respectively. Also as a side product of this work, we report significant improvements over state-of-the-art results in the task of next-frame prediction.

本研究使用自监督学习辅助提高CNN-based光流估计在实际情境中的性能，在合成和真实场景下混合训练神经网络并采用样本多变量多任务结构，以预测next-flow代替当前光流估计，并取得了较好的性能，实验结果表明总体改善了在实际KITTl基准测试中的光流估计。

光流和下一帧预测的混合学习，提升野外场景的光流表现