We present a learning-based framework, recurrent transformer network (RTN),
to restore heavily degraded old films. Instead of performing frame-wise
restoration, our method is based on the hidden knowledge learned from adjacent
frames that contain abundant information about the occlusion, which is
beneficial to restore challenging artifacts of each frame while ensuring
temporal coherency. Moreover, contrasting the representation of the current
frame and the hidden knowledge makes it possible to infer the scratch position
in an unsupervised manner, and such defect localization generalizes well to
real-world degradations. To better resolve mixed degradation and compensate for
the flow estimation error during frame alignment, we propose to leverage more
expressive transformer blocks for spatial restoration. Experiments on both
synthetic dataset and real-world old films demonstrate the significant
superiority of the proposed RTN over existing solutions. In addition, the same
framework can effectively propagate the color from keyframes to the whole
video, ultimately yielding compelling restored films. The implementation and
model will be released at
this https URL

本文提出了一种学习框架 —— 循环 Transformer 网络 (RTN)，用于修复破损的旧电影。该方法基于从旁边的帧中学习到的隐藏知识来进行恢复，能够有效推断划痕位置，同时保证时间连续性，利用更强大的 Transformer 块进行空间恢复，实验表明该方法优于现有的解决方案，并且可以有效地传播关键帧的颜色，最终呈现出令人信服的恢复电影。