Videos typically record the streaming and continuous visual data as discrete
consecutive frames. Since the storage cost is expensive for videos of high
fidelity, most of them are stored in a relatively low resolution and frame
rate. Recent works of Space-Time Video Super-Resolution (STVSR) are developed
to incorporate temporal interpolation and spatial super-resolution in a unified
framework. However, most of them only support a fixed up-sampling scale, which
limits their flexibility and applications. In this work, instead of following
the discrete representations, we propose Video Implicit Neural Representation
(VideoINR), and we show its applications for STVSR. The learned implicit neural
representation can be decoded to videos of arbitrary spatial resolution and
frame rate. We show that VideoINR achieves competitive performances with
state-of-the-art STVSR methods on common up-sampling scales and significantly
outperforms prior works on continuous and out-of-training-distribution scales.
Our project page is at this http URL .

本研究提出了 Video Implicit Neural Representation (VideoINR) 的框架，支持任意空间分辨率和帧率的 STVSR，相比之前的方法在超出训练分布范围的数据上表现出更好的性能，这对提高视频质量有着很大的实际意义。

VideoINR：学习视频隐式神经表示实现连续的时空超分辨率

VideoINR: Learning Video Implicit Neural Representation for Continuous  Space-Time Super-Resolution

Light field (LF) cameras record both intensity and directions of light rays,
and encode 3D scenes into 4D LF images. Recently, many convolutional neural
networks (CNNs) have been proposed for various LF image processing tasks.
However, it is challenging for CNNs to effectively process LF images since the
spatial and angular information are highly inter-twined with varying
disparities. In this paper, we propose a generic mechanism to disentangle these
coupled information for LF image processing. Specifically, we first design a
class of domain-specific convolutions to disentangle LFs from different
dimensions, and then leverage these disentangled features by designing
task-specific modules. Our disentangling mechanism can well incorporate the LF
structure prior and effectively handle 4D LF data. Based on the proposed
mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp)
for spatial super-resolution, angular super-resolution and disparity
estimation. Experimental results show that our networks achieve
state-of-the-art performance on all these three tasks, which demonstrates the
effectiveness, efficiency, and generality of our disentangling mechanism.
Project page: this https URL

本文介绍了一种用于轻场摄像机数据处理的卷积神经网络，利用分离机制处理空间和角度信息间的交织以实现空间分辨率的提升，角度超分辨率和视差估计。实验结果表明，该方法在所有三个任务实现了最好的表现。

光场分解用于超分辨率和视差估计

Disentangling Light Fields for Super-Resolution and Disparity Estimation

Light field (LF) images acquired by hand-held devices usually suffer from low
spatial resolution as the limited sampling resources have to be shared with the
angular dimension. LF spatial super-resolution (SR) thus becomes an
indispensable part of the LF camera processing pipeline. The
high-dimensionality characteristic and complex geometrical structure of LF
images make the problem more challenging than traditional single-image SR. The
performance of existing methods is still limited as they fail to thoroughly
explore the coherence among LF views and are insufficient in accurately
preserving the parallax structure of the scene. In this paper, we propose a
novel learning-based LF spatial SR framework, in which each view of an LF image
is first individually super-resolved by exploring the complementary information
among views with combinatorial geometry embedding. For accurate preservation of
the parallax structure among the reconstructed views, a regularization network
trained over a structure-aware loss function is subsequently appended to
enforce correct parallax relationships over the intermediate estimation. Our
proposed approach is evaluated over datasets with a large number of testing
images including both synthetic and real-world scenes. Experimental results
demonstrate the advantage of our approach over state-of-the-art methods, i.e.,
our method not only improves the average PSNR by more than 1.0 dB but also
preserves more accurate parallax details, at a lower computational cost.

本文提出一种基于学习的轻场景图像空间超分辨率框架，在每个轻场景图像视角学习探索不同的信息，并通过正则化网络结构来确保场景的正确视差关系，实验结果表明该方法不仅提高了平均峰值信噪比超过 1.0 dB，而且以更低的计算成本保持了更准确的视差细节。