Temporal convolutional networks (TCNs) are a commonly used architecture for temporal video segmentation. TCNs however, tend to suffer from over-segmentation errors and require additional refinement modules to ensure smoothness and temporal coherency. In this work, we propose a novel temporal encoder-decoder to tackle the problem of sequence fragmentation. In particular, the decoder follows a coarse-to-fine structure with an implicit ensemble of multiple temporal resolutions. The ensembling produces smoother segmentations that are more accurate and better-calibrated, bypassing the need for additional refinement modules. In addition, we enhance our training with a multi-resolution feature-augmentation strategy to promote robustness to varying temporal resolutions. Finally, to support our architecture and encourage further sequence coherency, we propose an action loss that penalizes misclassifications at the video level. Experiments show that our stand-alone architecture, together with our novel feature-augmentation strategy and new loss, outperforms the state-of-the-art on three temporal video segmentation benchmarks.

本文提出了一种新的带有多分辨率的粗到细结构解码器来处理序列断裂问题，使用多分辨率特征增强策略增强训练来提高对于不同时间分辨率的鲁棒性，同时提出了一个视频级别的行为损失来支持架构和鼓励序列的连贯性，在三个时间视频分割基准测试中打破了现有技术的记录。

粗到细的多分辨率时间卷积网络