Recent studies made great progress in video matting by extending the success
of trimap-based image matting to the video domain. In this paper, we push this
task toward a more practical setting and propose One-Trimap Video Matting
network (OTVM) that performs video matting robustly using only one
user-annotated trimap. A key of OTVM is the joint modeling of trimap
propagation and alpha prediction. Starting from baseline trimap propagation and
alpha prediction networks, our OTVM combines the two networks with an
alpha-trimap refinement module to facilitate information flow. We also present
an end-to-end training strategy to take full advantage of the joint model. Our
joint modeling greatly improves the temporal stability of trimap propagation
compared to the previous decoupled methods. We evaluate our model on two latest
video matting benchmarks, Deep Video Matting and VideoMatting108, and
outperform state-of-the-art by significant margins (MSE improvements of 56.4%
and 56.7%, respectively). The source code and model are available online:
this https URL

本文提出了一种 One-Trimap 视频抠图网络 (OTVM)，该网络能够在只使用一个用户注释的 trimap 的情况下，鲁棒地执行视频抠图，并通过联合建模 trimap 传播和 alpha 预测来实现。其中，alpha 预测与 trimap 传播网络以及 alpha-trimap 细化模块的结合，极大地提高了模型的时序稳定性，而采用的端到端训练策略，则能充分发挥联合模型的优势，创造出了在两个最新视频抠图基准上的最佳表现。

一步式三分图视频抠像

One-Trimap Video Matting

Natural image matting separates the foreground from background in fractional
occupancy which can be caused by highly transparent objects, complex foreground
(e.g., net or tree), and/or objects containing very fine details (e.g., hairs).
Although conventional matting formulation can be applied to all of the above
cases, no previous work has attempted to reason the underlying causes of
matting due to various foreground semantics.
We show how to obtain better alpha mattes by incorporating into our framework
semantic classification of matting regions. Specifically, we consider and learn
20 classes of matting patterns, and propose to extend the conventional trimap
to semantic trimap. The proposed semantic trimap can be obtained automatically
through patch structure analysis within trimap regions. Meanwhile, we learn a
multi-class discriminator to regularize the alpha prediction at semantic level,
and content-sensitive weights to balance different regularization losses.
Experiments on multiple benchmarks show that our method outperforms other
methods and has achieved the most competitive state-of-the-art performance.
Finally, we contribute a large-scale Semantic Image Matting Dataset with
careful consideration of data balancing across different semantic classes.

本文提出使用语义分类的前景语境信息以及语义 trimap 对自然图像抠图进行改进，并就此进行了大规模实验，获得了当下具有最高竞争力的性能表现。同时，我们提供了一个考虑不同语义类别平衡的大型语义图像抠图数据集。