Recently, Vision Transformer has achieved great success in recovering missing
details in low-resolution sequences, i.e., the video super-resolution (VSR)
task.Despite its superiority in VSR accuracy, the heavy computational burden as
well as the large memory footprint hinder the deployment of Transformer-based
VSR models on constrained devices.In this paper, we address the above issue by
proposing a novel feature-level masked processing framework: VSR with Masked
Intra and inter frame Attention (MIA-VSR).The core of MIA-VSR is leveraging
feature-level temporal continuity between adjacent frames to reduce redundant
computations and make more rational use of previously enhanced SR features.
Concretely, we propose an intra-frame and inter-frame attention block which
takes the respective roles of past features and input features into
consideration and only exploits previously enhanced features to provide
supplementary information. In addition, an adaptive block-wise mask prediction
module is developed to skip unimportant computations according to feature
similarity between adjacent frames. We conduct detailed ablation studies to
validate our contributions and compare the proposed method with recent
state-of-the-art VSR approaches. The experimental results demonstrate that
MIA-VSR improves the memory and computation efficiency over state-of-the-art
methods, without trading off PSNR accuracy. The code is available at
this https URL

通过提出一种新颖的特征级遮蔽处理框架 MIA-VSR，本文解决了 Vision Transformer 在受限设备上应用的计算负担和内存占用过大的问题，并通过详细的消融研究验证了方法的有效性和与最新技术的比较结果。