In this work, we propose a technique to convert CNN models for semantic
segmentation of static images into CNNs for video data. We describe a warping
method that can be used to augment existing architectures with very little
extra computational cost. This module is called NetWarp and we demonstrate its
use for a range of network architectures. The main design principle is to use
optical flow of adjacent frames for warping internal network representations
across time. A key insight of this work is that fast optical flow methods can
be combined with many different CNN architectures for improved performance and
end-to-end training. Experiments validate that the proposed approach incurs
only little extra computational cost, while improving performance, when video
streams are available. We achieve new state-of-the-art results on the CamVid
and Cityscapes benchmark datasets and show consistent improvements over
different baseline networks. Our code and models will be available at
this http URL

本文提出了一种将用于静态图像的 CNN 模型转换为用于视频数据的 CNN 模型的技术，利用名为 NetWarp 的模块，通过邻帧的光流在时间上进行网络表示的扭曲，同时将光流方法与不同的 CNN 架构结合以实现改进，最终在 CamVid 和 Cityscapes 数据集上实现了最新的最佳效果。