This paper addresses the task of space-time video super-resolution (ST-VSR).
Existing methods generally suffer from inaccurate motion estimation and motion
compensation (MEMC) problems for large motions. Inspired by recent progress in
physics-informed neural networks, we model the challenges of MEMC in ST-VSR as
a mapping between two continuous function spaces. Specifically, our approach
transforms independent low-resolution representations in the coarse-grained
continuous function space into refined representations with enriched
spatiotemporal details in the fine-grained continuous function space. To
achieve efficient and accurate MEMC, we design a Galerkin-type attention
function to perform frame alignment and temporal interpolation. Due to the
linear complexity of the Galerkin-type attention mechanism, our model avoids
patch partitioning and offers global receptive fields, enabling precise
estimation of large motions. The experimental results show that the proposed
method surpasses state-of-the-art techniques in both fixed-size and continuous
space-time video super-resolution tasks.

通过引入物理信息的神经网络，本文提出了解决空时视频超分辨率问题的方法，该方法能够准确处理大运动中的运动估计和运动补偿问题，并在固定大小和连续空时视频超分辨率任务中超过同类技术。

时空视频超分辨率与神经算子

Space-Time Video Super-resolution with Neural Operator

We propose Super-resolution Neural Operator (SRNO), a deep operator learning
framework that can resolve high-resolution (HR) images at arbitrary scales from
the low-resolution (LR) counterparts. Treating the LR-HR image pairs as
continuous functions approximated with different grid sizes, SRNO learns the
mapping between the corresponding function spaces. From the perspective of
approximation theory, SRNO first embeds the LR input into a higher-dimensional
latent representation space, trying to capture sufficient basis functions, and
then iteratively approximates the implicit image function with a kernel
integral mechanism, followed by a final dimensionality reduction step to
generate the RGB representation at the target coordinates. The key
characteristics distinguishing SRNO from prior continuous SR works are: 1) the
kernel integral in each layer is efficiently implemented via the Galerkin-type
attention, which possesses non-local properties in the spatial domain and
therefore benefits the grid-free continuum; and 2) the multilayer attention
architecture allows for the dynamic latent basis update, which is crucial for
SR problems to "hallucinate" high-frequency information from the LR image.
Experiments show that SRNO outperforms existing continuous SR methods in terms
of both accuracy and running time. Our code is at
this https URL

提出了一种深度操作学习框架 “SRNO”，它通过将低分辨率图像对作为使用不同网格大小逼近的连续函数来实现将高分辨率图像从低分辨率图像中恢复，其中关键特点是使用 Galerkin-type 注意力的内核积分和多层注意力结构的动态潜在基础更新。