Semantic segmentation requires large amounts of pixel-wise annotations to
learn accurate models. In this paper, we present a video prediction-based
methodology to scale up training sets by synthesizing new training samples in
order to improve the accuracy of semantic segmentation networks. We exploit
video prediction models' ability to predict future frames in order to also
predict future labels. A joint propagation strategy is also proposed to
alleviate mis-alignments in synthesized samples. We demonstrate that training
segmentation models on datasets augmented by the synthesized samples leads to
significant improvements in accuracy. Furthermore, we introduce a novel
boundary label relaxation technique that makes training robust to annotation
noise and propagation artifacts along object boundaries. Our proposed methods
achieve state-of-the-art mIoUs of 83.5% on Cityscapes and 82.9% on CamVid. Our
single model, without model ensembles, achieves 72.8% mIoU on the KITTI
semantic segmentation test set, which surpasses the winning entry of the ROB
challenge 2018. Our code and videos can be found at
this https URL

通过视频预测方法合成新的训练样本并引入边界标签松弛技术，使模型对标注噪声和传播伪影更加鲁棒，实现在 Cityscapes 数据集上 83.5%、CamVid 上 82.9% 的 mIoUs 并在 KITTI 语义分割测试集上取得 72.8% 的 mIoU，超过 ROB 挑战 2018 年的获奖模型。