Compressed video super-resolution (VSR) aims to restore high-resolution
frames from compressed low-resolution counterparts. Most recent VSR approaches
often enhance an input frame by borrowing relevant textures from neighboring
video frames. Although some progress has been made, there are grand challenges
to effectively extract and transfer high-quality textures from compressed
videos where most frames are usually highly degraded. In this paper, we propose
a novel Frequency-Transformer for compressed video super-resolution (FTVSR)
that conducts self-attention over a joint space-time-frequency domain. First,
we divide a video frame into patches, and transform each patch into DCT
spectral maps in which each channel represents a frequency band. Such a design
enables a fine-grained level self-attention on each frequency band, so that
real visual texture can be distinguished from artifacts, and further utilized
for video frame restoration. Second, we study different self-attention schemes,
and discover that a divided attention which conducts a joint space-frequency
attention before applying temporal attention on each frequency band, leads to
the best video enhancement quality. Experimental results on two widely-used
video super-resolution benchmarks show that FTVSR outperforms state-of-the-art
approaches on both uncompressed and compressed videos with clear visual
margins. Code is available at this https URL

本文提出了一种基于 Frequency-Transformer 的压缩视频超分辨率方法，通过在空间、时间、频率域进行自注意力机制，有效地提取并转移高质量纹理进行视频帧增强和恢复。实验结果表明，该方法在两个广泛使用的视频超分辨率基准测试中均表现优于现有技术，并且在无损和有损视频上均具有明显的视觉优势。