We introduce a compact network for holistic scene flow estimation, called SENSE, which shares common encoder features among four closely-related tasks: optical flow estimation, disparity estimation from stereo, occlusion estimation, and semantic segmentation. Our key insight is that sharing features makes the network more compact, induces better feature representations, and can better exploit interactions among these tasks to handle partially labeled data. With a shared encoder, we can flexibly add decoders for different tasks during training. This modular design leads to a compact and efficient model at inference time. Exploiting the interactions among these tasks allows us to introduce distillation and self-supervised losses in addition to supervised losses, which can better handle partially labeled real-world data. SENSE achieves state-of-the-art results on several optical flow benchmarks and runs as fast as networks specifically designed for optical flow. It also compares favorably against the state of the art on stereo and scene flow, while consuming much less memory.

介绍了一种名为SENSE的全局场景流的紧凑网络，通过共享编码器特征在光流、立体视角差、遮挡与语义分割四个紧密相关任务中进行估计，并利用多任务学习得到更好的特征表示，可处理部分标记数据。该模型在多个光流基准测试上取得了最先进的结果，并且具有与专门设计用于光流的网络一样快的运行速度，同时消耗更少的内存。

SENSE: 一种用于场景流估计的共享编码器网络