As a critical clue of video super-resolution (VSR), inter-frame alignment
significantly impacts overall performance. However, accurate pixel-level
alignment is a challenging task due to the intricate motion interweaving in the
video. In response to this issue, we introduce a novel paradigm for VSR named
\textbf{Semantic Lens}, predicated on semantic priors drawn from degraded
videos. Specifically, video is modeled as instances, events, and scenes via a
Semantic Extractor. Those semantics assist the Pixel Enhancer in understanding
the recovered contents and generating more realistic visual results. The
distilled global semantics embody the scene information of each frame, while
the instance-specific semantics assemble the spatial-temporal contexts related
to each instance. Furthermore, we devise a \textbf{S}emantics-\textbf{P}owered
\textbf{A}ttention \textbf{C}ross-\textbf{E}mbedding (SPACE) block to bridge
the pixel-level features with semantic knowledge, composed of a \textbf{G}lobal
\textbf{P}erspective \textbf{S}hifter (GPS) and an \textbf{I}nstance-Specific
\textbf{S}emantic \textbf{E}mbedding \textbf{E}ncoder (ISEE). Concretely, the
GPS module generates pairs of affine transformation parameters for pixel-level
feature modulation conditioned on global semantics. After that, the ISEE module
harnesses the attention mechanism to align the adjacent frames in the
instance-centric semantic space. In addition, we incorporate a simple yet
effective pre-alignment module to alleviate the difficulty of model training.
Extensive experiments demonstrate the superiority of our model over existing
state-of-the-art VSR methods.

该研究提出了一种基于视频语义的超分辨率方法，通过引入语义先验信息并结合像素级特征，实现了准确的帧间对齐，从而在视频超分辨率任务中获得了优于现有方法的结果。