We propose a flow-guided transformer, which innovatively leverage the motion
discrepancy exposed by optical flows to instruct the attention retrieval in
transformer for high fidelity video inpainting. More specially, we design a
novel flow completion network to complete the corrupted flows by exploiting the
relevant flow features in a local temporal window. With the completed flows, we
propagate the content across video frames, and adopt the flow-guided
transformer to synthesize the rest corrupted regions. We decouple transformers
along temporal and spatial dimension, so that we can easily integrate the
locally relevant completed flows to instruct spatial attention only.
Furthermore, we design a flow-reweight module to precisely control the impact
of completed flows on each spatial transformer. For the sake of efficiency, we
introduce window partition strategy to both spatial and temporal transformers.
Especially in spatial transformer, we design a dual perspective spatial MHSA,
which integrates the global tokens to the window-based attention. Extensive
experiments demonstrate the effectiveness of the proposed method qualitatively
and quantitatively. Codes are available at this https URL

提出了一种基于光学流的流程引导变压器，用于高保真视频修复，通过设计新颖的流完整网络来完成损坏的光流，并使用流引导变压器来合成其余的损坏区域，通过时间和空间维度解耦变压器，以便仅集成局部相关的完成的流来控制流对每个空间变压器的影响。