Recent work on dense optical flow has shown significant progress, primarily
in a supervised learning manner requiring a large amount of labeled data. Due
to the expensiveness of obtaining large scale real-world data, computer
graphics are typically leveraged for constructing datasets. However, there is a
common belief that synthetic-to-real domain gaps limit generalization to real
scenes. In this paper, we show that the required characteristics in an optical
flow dataset are rather simple and present a simpler synthetic data generation
method that achieves a certain level of realism with compositions of elementary
operations. With 2D motion-based datasets, we systematically analyze the
simplest yet critical factors for generating synthetic datasets. Furthermore,
we propose a novel method of utilizing occlusion masks in a supervised method
and observe that suppressing gradients on occluded regions serves as a powerful
initial state in the curriculum learning sense. The RAFT network initially
trained on our dataset outperforms the original RAFT on the two most
challenging online benchmarks, MPI Sintel and KITTI 2015.

最近对于密集光流的研究取得了显著进展，主要以需要大量标记数据的监督学习方式为基础。然而，由于获取大规模真实世界数据的昂贵性，通常会利用计算机图形来构建数据集。本文展示了在光流数据集中所需的特征是相当简单的，并提出了一种更简单的合成数据生成方法，通过基本操作的组合实现了一定水平的真实感。通过对 2D 运动数据集的系统分析，我们进一步介绍了生成合成数据集最简单但至关重要的因素。此外，我们提出了一种新的方法，在监督学习中利用遮挡掩模，并观察到抑制遮挡区域梯度在课程学习意义上作为强大的初始状态。在我们的数据集上进行了 RAF 网络的初始训练，结果在 MPI Sintel 和 KITTI 2015 这两个最具挑战性的在线基准测试中超过了原始的 RAF 网络。

细节中的魔鬼：简单高效的光流合成数据生成

The Devil in the Details: Simple and Effective Optical Flow Synthetic  Data Generation

In this paper we propose a novel approach to estimate dense optical flow from
sparse lidar data acquired on an autonomous vehicle. This is intended to be
used as a drop-in replacement of any image-based optical flow system when
images are not reliable due to e.g. adverse weather conditions or at night. In
order to infer high resolution 2D flows from discrete range data we devise a
three-block architecture of multiscale filters that combines multiple
intermediate objectives, both in the lidar and image domain. To train this
network we introduce a dataset with approximately 20K lidar samples of the
Kitti dataset which we have augmented with a pseudo ground-truth image-based
optical flow computed using FlowNet2. We demonstrate the effectiveness of our
approach on Kitti, and show that despite using the low-resolution and sparse
measurements of the lidar, we can regress dense optical flow maps which are at
par with those estimated with image-based methods.

本文提出了一种新的方法，从自动驾驶车辆的稀疏激光雷达数据中估计密集的光流，用作图像不可靠的情况下，如恶劣天气或夜间，替代任何基于图像的光流系统的替代方案。该方法使用多尺度滤波器的三级结构来推断离散范围数据的高分辨率 2D 流，并在 lidar 和图像域中结合多个中间目标。通过使用 FlowNet2 计算的假伪基于图像的光流，我们介绍了一个包含约 20K lidar 样本的 Kitti 数据集，并使用该数据集训练网络。我们在 Kitti 数据集上演示了我们方法的有效性，并表明尽管使用低分辨率和稀疏的 lidar 测量，我们可以回归与基于图像的方法相当的密集光流图。

基于稀疏雷达的自动驾驶车辆密集光流幻像

Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous  Vehicles

This paper shows how to extract dense optical flow from videos with a
convolutional neural network (CNN). The proposed model constitutes a potential
building block for deeper architectures to allow using motion without resorting
to an external algorithm, \eg for recognition in videos. We derive our network
architecture from signal processing principles to provide desired invariances
to image contrast, phase and texture. We constrain weights within the network
to enforce strict rotation invariance and substantially reduce the number of
parameters to learn. We demonstrate end-to-end training on only 8 sequences of
the Middlebury dataset, orders of magnitude less than competing CNN-based
motion estimation methods, and obtain comparable performance to classical
methods on the Middlebury benchmark. Importantly, our method outputs a
distributed representation of motion that allows representing multiple,
transparent motions, and dynamic textures. Our contributions on network design
and rotation invariance offer insights nonspecific to motion estimation.

本文提出了一种卷积神经网络用于从视频中提取密集的光流，旨在为深度架构构建潜在的模块，以允许在视频中使用运动而无需借助外部算法，通过考虑信号处理原则构建网络结构，强制 “旋转不变性”，并提供一种分布式表示运动的方法。