Blind-spot networks (BSN) have been prevalent network architectures in
self-supervised image denoising (SSID). Existing BSNs are mostly conducted with
convolution layers. Although transformers offer potential solutions to the
limitations of convolutions and have demonstrated success in various image
restoration tasks, their attention mechanisms may violate the blind-spot
requirement, thus restricting their applicability in SSID. In this paper, we
present a transformer-based blind-spot network (TBSN) by analyzing and
redesigning the transformer operators that meet the blind-spot requirement.
Specifically, TBSN follows the architectural principles of dilated BSNs, and
incorporates spatial as well as channel self-attention layers to enhance the
network capability. For spatial self-attention, an elaborate mask is applied to
the attention matrix to restrict its receptive field, thus mimicking the
dilated convolution. For channel self-attention, we observe that it may leak
the blind-spot information when the channel number is greater than spatial size
in the deep layers of multi-scale architectures. To eliminate this effect, we
divide the channel into several groups and perform channel attention
separately. Furthermore, we introduce a knowledge distillation strategy that
distills TBSN into smaller denoisers to improve computational efficiency while
maintaining performance. Extensive experiments on real-world image denoising
datasets show that TBSN largely extends the receptive field and exhibits
favorable performance against state-of-the-art SSID methods. The code and
pre-trained models will be publicly available at
this https URL

本文提出了基于 transformer 的盲点网络（TBSN），通过分析和重新设计满足盲点要求的 transformer 操作符，扩展了其感受野，实现了有利于自监督图像去噪（SSID）任务处理的效果，并引入知识蒸馏策略提高计算效率。

TBSN: 基于 Transformer 的自监督图像降噪的盲点网络

TBSN: Transformer-Based Blind-Spot Network for Self-Supervised Image  Denoising

Significant progress has been made in self-supervised image denoising (SSID)
in the recent few years. However, most methods focus on dealing with spatially
independent noise, and they have little practicality on real-world sRGB images
with spatially correlated noise. Although pixel-shuffle downsampling has been
suggested for breaking the noise correlation, it breaks the original
information of images, which limits the denoising performance. In this paper,
we propose a novel perspective to solve this problem, i.e., seeking for
spatially adaptive supervision for real-world sRGB image denoising.
Specifically, we take into account the respective characteristics of flat and
textured regions in noisy images, and construct supervisions for them
separately. For flat areas, the supervision can be safely derived from
non-adjacent pixels, which are much far from the current pixel for excluding
the influence of the noise-correlated ones. And we extend the blind-spot
network to a blind-neighborhood network (BNN) for providing supervision on flat
areas. For textured regions, the supervision has to be closely related to the
content of adjacent pixels. And we present a locally aware network (LAN) to
meet the requirement, while LAN itself is selectively supervised with the
output of BNN. Combining these two supervisions, a denoising network (e.g.,
U-Net) can be well-trained. Extensive experiments show that our method performs
favorably against state-of-the-art SSID methods on real-world sRGB photographs.
The code is available at this https URL

本研究提出一种新的思路来解决现实世界 sRGB 图像去噪中空间相关噪声的问题，其通过分别考虑噪声图像中平滑区域和纹理区域的特性，并构建相应的监督来训练模型，最终得到了更好的去噪效果。