We present a joint learning scheme of video super-resolution and deblurring,
called VSRDB, to restore clean high-resolution (HR) videos from blurry
low-resolution (LR) ones. This joint restoration problem has drawn much less
attention compared to single restoration problems. In this paper, we propose a
novel flow-guided dynamic filtering (FGDF) and iterative feature refinement
with multi-attention (FRMA), which constitutes our VSRDB framework, denoted as
FMA-Net. Specifically, our proposed FGDF enables precise estimation of both
spatio-temporally-variant degradation and restoration kernels that are aware of
motion trajectories through sophisticated motion representation learning.
Compared to conventional dynamic filtering, the FGDF enables the FMA-Net to
effectively handle large motions into the VSRDB. Additionally, the stacked FRMA
blocks trained with our novel temporal anchor (TA) loss, which temporally
anchors and sharpens features, refine features in a course-to-fine manner
through iterative updates. Extensive experiments demonstrate the superiority of
the proposed FMA-Net over state-of-the-art methods in terms of both
quantitative and qualitative quality. Codes and pre-trained models are
available at: this https URL

通过联合学习视频超分辨率和去模糊的方法，本文提出了一种名为 VSRDB 的视频清晰恢复方案，以从模糊的低分辨率视频中还原出干净的高分辨率视频。通过 FGDF 和 FRMA 两个部分构成的 FMA-Net 框架，我们的方法能够精确估计时空变化的退化和恢复内核，并通过迭代特征细化的方式进行多注意力的特征改进。鲁棒的实验证明了本文提出的 FMA-Net 在定量和定性方面优于现有的方法。

FMA-Net: 基于流向引导的动态滤波和多注意力迭代特征精炼的联合视频超分辨率和去模糊

FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement  with Multi-Attention for Joint Video Super-Resolution and Deblurring

Residual mappings have been shown to perform representation learning in the
first layers and iterative feature refinement in higher layers. This interplay,
combined with their stabilizing effect on the gradient norms, enables them to
train very deep networks. In this paper, we take a step further and introduce
entangled residual mappings to generalize the structure of the residual
connections and evaluate their role in iterative learning representations. An
entangled residual mapping replaces the identity skip connections with
specialized entangled mappings such as orthogonal, sparse, and structural
correlation matrices that share key attributes (eigenvalues, structure, and
Jacobian norm) with identity mappings. We show that while entangled mappings
can preserve the iterative refinement of features across various deep models,
they influence the representation learning process in convolutional networks
differently than attention-based models and recurrent neural networks. In
general, we find that for CNNs and Vision Transformers entangled sparse mapping
can help generalization while orthogonal mappings hurt performance. For
recurrent networks, orthogonal residual mappings form an inductive bias for
time-variant sequences, which degrades accuracy on time-invariant tasks.

本文提出了纠缠残差映射来推广残差连接的结构，并评估它们在迭代学习表示方面的作用。研究表明，在 CNN 和 Vision Transformer 中，纠缠稀疏映射可以帮助泛化，而正交映射会损害性能。在循环网络方面，正交残差映射形成了一种对时间变化序列的归纳偏差，这会降低处理时间不变的任务的准确性。