Images or videos captured by the Under-Display Camera (UDC) suffer from
severe degradation, such as saturation degeneration and color shift. While
restoration for UDC has been a critical task, existing works of UDC restoration
focus only on images. UDC video restoration (UDC-VR) has not been explored in
the community. In this work, we first propose a GAN-based generation pipeline
to simulate the realistic UDC degradation process. With the pipeline, we build
the first large-scale UDC video restoration dataset called PexelsUDC, which
includes two subsets named PexelsUDC-T and PexelsUDC-P corresponding to
different displays for UDC. Using the proposed dataset, we conduct extensive
benchmark studies on existing video restoration methods and observe their
limitations on the UDC-VR task. To this end, we propose a novel
transformer-based baseline method that adaptively enhances degraded videos. The
key components of the method are a spatial branch with local-aware
transformers, a temporal branch embedded temporal transformers, and a
spatial-temporal fusion module. These components drive the model to fully
exploit spatial and temporal information for UDC-VR. Extensive experiments show
that our method achieves state-of-the-art performance on PexelsUDC. The
benchmark and the baseline method are expected to promote the progress of
UDC-VR in the community, which will be made public.

基于 GAN 的生成流程模拟了 UDC 的退化过程，创建了名为 PexelsUDC 的大规模数据集，通过该数据集对现有的视频修复方法进行了广泛的基准研究，并提出了一种基于 transformer 的基线方法，该方法在 PexelsUDC 上取得了最先进的性能。