The most recent efforts in video matting have focused on eliminating trimap
dependency since trimap annotations are expensive and trimap-based methods are
less adaptable for real-time applications. Despite the latest tripmap-free
methods showing promising results, their performance often degrades when
dealing with highly diverse and unstructured videos. We address this limitation
by introducing Adaptive Matting for Dynamic Videos, termed AdaM, which is a
framework designed for simultaneously differentiating foregrounds from
backgrounds and capturing alpha matte details of human subjects in the
foreground. Two interconnected network designs are employed to achieve this
goal: (1) an encoder-decoder network that produces alpha mattes and
intermediate masks which are used to guide the transformer in adaptively
decoding foregrounds and backgrounds, and (2) a transformer network in which
long- and short-term attention combine to retain spatial and temporal contexts,
facilitating the decoding of foreground details. We benchmark and study our
methods on recently introduced datasets, showing that our model notably
improves matting realism and temporal coherence in complex real-world videos
and achieves new best-in-class generalizability. Further details and examples
are available at this https URL

通过使用编码解码网络和 transformer 网络，本文介绍了一个名为 AdaM 的自适应合成框架，用于动态视频中的前景和背景分割及 alpha 通道遮罩；在多数据集测试中，AdaM 显示出较高的适应性和合成实况，同时取得新的最佳通用性。