Human matting is a foundation task in image and video processing, where human
foreground pixels are extracted from the input. Prior works either improve the
accuracy by additional guidance or improve the temporal consistency of a single
instance across frames. We propose a new framework MaGGIe, Masked Guided
Gradual Human Instance Matting, which predicts alpha mattes progressively for
each human instances while maintaining the computational cost, precision, and
consistency. Our method leverages modern architectures, including transformer
attention and sparse convolution, to output all instance mattes simultaneously
without exploding memory and latency. Although keeping constant inference costs
in the multiple-instance scenario, our framework achieves robust and versatile
performance on our proposed synthesized benchmarks. With the higher quality
image and video matting benchmarks, the novel multi-instance synthesis approach
from publicly available sources is introduced to increase the generalization of
models in real-world scenarios.

我们提出了一种新的框架 MaGGIe，即遮蔽引导逐渐人物实例抠图（Masked Guided Gradual Human Instance Matting），该框架能够在保持计算成本、精确度和一致性的同时逐步预测每个人物实例的透明度抠图。我们的方法利用现代架构，如 Transformer 注意力和稀疏卷积，同时输出所有实例的抠图，而不会发生内存和延迟问题。尽管在多实例情景下保持恒定的推理成本，我们的框架在我们提出的综合基准测试中实现了稳健而灵活的性能。通过使用更高质量的图像和视频抠图基准，引入了来自公开可用源的新型多实例综合方法，以增加模型在现实世界场景中的泛化能力。

MaGGIe: 掩码引导渐进式人体实例抠图

MaGGIe: Masked Guided Gradual Human Instance Matting

This paper presents a new practical training method for human matting, which
demands delicate pixel-level human region identification and significantly
laborious annotations. To reduce the annotation cost, most existing matting
approaches often rely on image synthesis to augment the dataset. However, the
unnaturalness of synthesized training images brings in a new domain
generalization challenge for natural images. To address this challenge, we
introduce a new learning paradigm, weakly semi-supervised human matting
(WSSHM), which leverages a small amount of expensive matte labels and a large
amount of budget-friendly segmentation labels, to save the annotation cost and
resolve the domain generalization problem. To achieve the goal of WSSHM, we
propose a simple and effective training method, named Matte Label Blending
(MLB), that selectively guides only the beneficial knowledge of the
segmentation and matte data to the matting model. Extensive experiments with
our detailed analysis demonstrate our method can substantially improve the
robustness of the matting model using a few matte data and numerous
segmentation data. Our training method is also easily applicable to real-time
models, achieving competitive accuracy with breakneck inference speed (328 FPS
on NVIDIA V100 GPU). The implementation code is available at
https://github.com/clovaai/WSSHM.

本研究提出了一种新的实用人像抠图训练方法，通过精细的像素级人体区域识别和大量繁重的标注来实现。为了降低标注成本，现有的抠图方法通常依赖于图像合成来增加数据集。然而，合成训练图像的不自然性给自然图像带来了新的领域泛化挑战。为了解决这个挑战，我们引入了一种新的学习范式，弱半监督人像抠图（WSSHM），它利用少量昂贵的抠图标签和大量经济实惠的分割标签来节省标注成本和解决领域泛化问题。为了实现 WSSHM 的目标，我们提出了一种简单有效的训练方法，称为抠图标签融合（MLB），仅选择性地将分割和抠图数据的有益知识引导到抠图模型。通过详细分析的大量实验表明，我们的方法能够在使用少量抠图数据和大量分割数据的情况下显著提高抠图模型的鲁棒性。我们的训练方法也适用于实时模型，并具有竞争力的准确性和极快的推理速度（在 NVIDIA V100 GPU 上为 328 FPS）。实现代码可在 https://github.com/clovaai/WSSHM 获得。

向标签有效的人物抠图迈进：无需辅助轮廓线的弱监督简易基线

Towards Label-Efficient Human Matting: A Simple Baseline for Weakly  Semi-Supervised Trimap-Free Human Matting

Human matting, high quality extraction of humans from natural images, is
crucial for a wide variety of applications. Since the matting problem is
severely under-constrained, most previous methods require user interactions to
take user designated trimaps or scribbles as constraints. This user-in-the-loop
nature makes them difficult to be applied to large scale data or time-sensitive
scenarios. In this paper, instead of using explicit user input constraints, we
employ implicit semantic constraints learned from data and propose an automatic
human matting algorithm (SHM). SHM is the first algorithm that learns to
jointly fit both semantic information and high quality details with deep
networks. In practice, simultaneously learning both coarse semantics and fine
details is challenging. We propose a novel fusion strategy which naturally
gives a probabilistic estimation of the alpha matte. We also construct a very
large dataset with high quality annotations consisting of 35,513 unique
foregrounds to facilitate the learning and evaluation of human matting.
Extensive experiments on this dataset and plenty of real images show that SHM
achieves comparable results with state-of-the-art interactive matting methods.

本文提出了一种自动的人像抠图算法（SHM），以数据学习的隐式语义约束为基础，利用深度卷积神经网络同时拟合语义信息和高质量细节，经实验证明，该算法效果与互动抠图方法相当，并构建了一个包含 35,513 个前景图的高质量注释数据集。