Recent years have seen remarkable progress in semantic segmentation. Yet, it remains a challenging task to apply segmentation techniques to video-based applications. Specifically, the high throughput of video streams, the sheer cost of running fully convolutional networks, together with the low-latency requirements in many real-world applications, e.g. autonomous driving, present a significant challenge to the design of the video segmentation framework. To tackle this combined challenge, we develop a framework for video semantic segmentation, which incorporates two novel components: (1) a feature propagation module that adaptively fuses features over time via spatially variant convolution, thus reducing the cost of per-frame computation; and (2) an adaptive scheduler that dynamically allocate computation based on accuracy prediction. Both components work together to ensure low latency while maintaining high segmentation quality. On both Cityscapes and CamVid, the proposed framework obtained competitive performance compared to the state of the art, while substantially reducing the latency, from 360 ms to 119 ms.

本研究提出了一种视频语义分割框架，该框架利用特征传播模块和自适应调度器相结合的方法，通过时间间隔自适应地融合特征并根据准确性预测动态分配计算量，以实现低延迟和高语义分割质量。实验结果表明，在与现有模型的比较中，该模型在Cityscapes和CamVid这两个数据集上获得了竞争性的性能，并将延迟从360 ms降低到119 ms。

低延迟视频语义分割