In this work, we explore whether modeling recurrence into the Transformer
architecture can both be beneficial and efficient, by building an extremely
simple recurrent module into the Transformer. We compare our model to baselines
following the training and evaluation recipe of BERT. Our results confirm that
recurrence can indeed improve Transformer models by a consistent margin,
without requiring low-level performance optimizations, and while keeping the
number of parameters constant. For example, our base model achieves an absolute
improvement of 2.1 points averaged across 10 tasks and also demonstrates
increased stability in fine-tuning over a range of learning rates.

本文研究在 Transformer 架构中引入循环模块是否可以提高性能，实验结果表明，引入循环模块可以提高 Transformer 模型的稳定性和性能，而不需要使用低层次的性能优化方法，并且参数数量保持不变。

简单递归改善遮蔽语言模型

Simple Recurrence Improves Masked Language Models

Existing works reduce motion blur and up-convert frame rate through two
separate ways, including frame deblurring and frame interpolation. However, few
studies have approached the joint video enhancement problem, namely
synthesizing high-frame-rate clear results from low-frame-rate blurry inputs.
In this paper, we propose a blurry video frame interpolation method to reduce
motion blur and up-convert frame rate simultaneously. Specifically, we develop
a pyramid module to cyclically synthesize clear intermediate frames. The
pyramid module features adjustable spatial receptive field and temporal scope,
thus contributing to controllable computational complexity and restoration
ability. Besides, we propose an inter-pyramid recurrent module to connect
sequential models to exploit the temporal relationship. The pyramid module
integrates a recurrent module, thus can iteratively synthesize temporally
smooth results without significantly increasing the model size. Extensive
experimental results demonstrate that our method performs favorably against
state-of-the-art methods.

本研究提出了一种模糊视频帧插值方法，通过金字塔模块和内部循环递推模块实现一定程度上同时减少运动模糊和提高帧率。实验证明，该方法在影像增强方面表现优异。