Under-Display Camera (UDC) is an emerging technology that achieves
full-screen display via hiding the camera under the display panel. However, the
current implementation of UDC causes serious degradation. The incident light
required for camera imaging undergoes attenuation and diffraction when passing
through the display panel, leading to various artifacts in UDC imaging.
Presently, the prevailing UDC image restoration methods predominantly utilize
convolutional neural network architectures, whereas Transformer-based methods
have exhibited superior performance in the majority of image restoration tasks.
This is attributed to the Transformer's capability to sample global features
for the local reconstruction of images, thereby achieving high-quality image
restoration. In this paper, we observe that when using the Vision Transformer
for UDC degraded image restoration, the global attention samples a large amount
of redundant information and noise. Furthermore, compared to the ordinary
Transformer employing dense attention, the Transformer utilizing sparse
attention can alleviate the adverse impact of redundant information and noise.
Building upon this discovery, we propose a Segmentation Guided Sparse
Transformer method (SGSFormer) for the task of restoring high-quality images
from UDC degraded images. Specifically, we utilize sparse self-attention to
filter out redundant information and noise, directing the model's attention to
focus on the features more relevant to the degraded regions in need of
reconstruction. Moreover, we integrate the instance segmentation map as prior
information to guide the sparse self-attention in filtering and focusing on the
correct regions.

在本文中，我们观察到在使用 Vision Transformer 进行 UDC 受损图像恢复时，全局注意机制会采样大量冗余信息和噪音。基于此发现，我们提出了一种基于分割引导的稀疏 Transformer 方法（SGSFormer），用于从 UDC 受损图像中恢复高质量图像。具体而言，我们利用稀疏自注意力过滤冗余信息和噪音，引导模型的注意力集中在需要重建的受损区域上。此外，我们还集成了实例分割图作为先验信息，以指导稀疏自注意力在过滤和关注正确区域方面的工作。