We investigate two crucial and closely related aspects of CNNs for optical
flow estimation: models and training. First, we design a compact but effective
CNN model, called PWC-Net, according to simple and well-established principles:
pyramidal processing, warping, and cost volume processing. PWC-Net is 17 times
smaller in size, 2 times faster in inference, and 11\% more accurate on Sintel
final than the recent FlowNet2 model. It is the winning entry in the optical
flow competition of the robust vision challenge. Next, we experimentally
analyze the sources of our performance gains. In particular, we use the same
training procedure of PWC-Net to retrain FlowNetC, a sub-network of FlowNet2.
The retrained FlowNetC is 56\% more accurate on Sintel final than the
previously trained one and even 5\% more accurate than the FlowNet2 model. We
further improve the training procedure and increase the accuracy of PWC-Net on
Sintel by 10\% and on KITTI 2012 and 2015 by 20\%. Our newly trained model
parameters and training protocols will be available on
this https URL

通过简单和成熟的原则，包括金字塔处理，扭曲和成本体积处理，我们设计了一种紧凑但有效的 CNN 模型 PWC-Net 进行光流估计，并通过相同的训练过程对 FlowNetC 进行重新训练，提高了 56% 的准确性，并进一步改进了训练过程，将 PWC-Net 在 Sintel 上的准确性提高了 10％，在 KITTI 2012 和 2015 上提高了 20％，该模型在 Robust Vision Challenge 的光流比赛中获胜。

模型重要，训练同样重要：一项用于光流估计的 CNN 实证研究

Models Matter, So Does Training: An Empirical Study of CNNs for Optical  Flow Estimation

We present a compact but effective CNN model for optical flow, called
PWC-Net. PWC-Net has been designed according to simple and well-established
principles: pyramidal processing, warping, and the use of a cost volume. Cast
in a learnable feature pyramid, PWC-Net uses the cur- rent optical flow
estimate to warp the CNN features of the second image. It then uses the warped
features and features of the first image to construct a cost volume, which is
processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in
size and easier to train than the recent FlowNet2 model. Moreover, it
outperforms all published optical flow methods on the MPI Sintel final pass and
KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024x436)
images. Our models are available on this https URL

我们提出了一种紧凑但有效的卷积神经网络模型，用于光流预测，名为 PWC-Net，其通过金字塔处理、扭曲变换和成本体积使用学习功能进行设计，该模型在 MPI Sintel 最终传递和 KITTI 2015 基准测试上优于所有发表的光流方法，同时比最近的 FlowNet2 模型小 17 倍且易于训练。