Flow-based propagation and spatiotemporal Transformer are two mainstream
mechanisms in video inpainting (VI). Despite the effectiveness of these
components, they still suffer from some limitations that affect their
performance. Previous propagation-based approaches are performed separately
either in the image or feature domain. Global image propagation isolated from
learning may cause spatial misalignment due to inaccurate optical flow.
Moreover, memory or computational constraints limit the temporal range of
feature propagation and video Transformer, preventing exploration of
correspondence information from distant frames. To address these issues, we
propose an improved framework, called ProPainter, which involves enhanced
ProPagation and an efficient Transformer. Specifically, we introduce
dual-domain propagation that combines the advantages of image and feature
warping, exploiting global correspondences reliably. We also propose a
mask-guided sparse video Transformer, which achieves high efficiency by
discarding unnecessary and redundant tokens. With these components, ProPainter
outperforms prior arts by a large margin of 1.46 dB in PSNR while maintaining
appealing efficiency.

通过改进的 ProPainter 框架，结合了增强的 ProPagation 和高效的 Transformer 机制，实现了视频修复的高效性和更好的效果。

ProPainter: 提升视频修复的传播和变换器

ProPainter: Improving Propagation and Transformer for Video Inpainting

Recently, the Transformer module has been transplanted from natural language
processing to computer vision. This paper applies the Transformer to
video-based person re-identification, where the key issue is to extract the
discriminative information from a tracklet. We show that, despite the strong
learning ability, the vanilla Transformer suffers from an increased risk of
over-fitting, arguably due to a large number of attention parameters and
insufficient training data. To solve this problem, we propose a novel pipeline
where the model is pre-trained on a set of synthesized video data and then
transferred to the downstream domains with the perception-constrained
Spatiotemporal Transformer (STT) module and Global Transformer (GT) module. The
derived algorithm achieves significant accuracy gain on three popular
video-based person re-identification benchmarks, MARS, DukeMTMC-VideoReID, and
LS-VID, especially when the training and testing data are from different
domains. More importantly, our research sheds light on the application of the
Transformer on highly-structured visual data.

本研究将 Transformer 应用于基于视频的人体再识别，提出一种新的基于预训练的模型，并使用感知约束的时空 Transformer 模块和全局 Transformer 模块转换到下游域，取得了显著的准确率提高。