Several recent studies advocate the use of spectral discriminators, which
evaluate the Fourier spectra of images for generative modeling. However, the
effectiveness of the spectral discriminators is not well interpreted yet. We
tackle this issue by examining the spectral discriminators in the context of
perceptual image super-resolution (i.e., GAN-based SR), as SR image quality is
susceptible to spectral changes. Our analyses reveal that the spectral
discriminator indeed performs better than the ordinary (a.k.a. spatial)
discriminator in identifying the differences in the high-frequency range;
however, the spatial discriminator holds an advantage in the low-frequency
range. Thus, we suggest that the spectral and spatial discriminators shall be
used simultaneously. Moreover, we improve the spectral discriminators by first
calculating the patch-wise Fourier spectrum and then aggregating the spectra by
Transformer. We verify the effectiveness of the proposed method twofold. On the
one hand, thanks to the additional spectral discriminator, our obtained SR
images have their spectra better aligned to those of the real images, which
leads to a better PD tradeoff. On the other hand, our ensembled discriminator
predicts the perceptual quality more accurately, as evidenced in the
no-reference image quality assessment task.

通过比较频谱鉴别器和空间鉴别器在高频和低频范围内的表现差异，我们建议同时使用频谱和空间鉴别器，并通过 Transformer 计算补丁的傅立叶谱并聚合谱来改进频谱鉴别器的效果。添加频谱鉴别器使得我们得到的超分辨率图像其频谱更接近真实图像，从而在感知细节和图像质量之间达到更好的平衡；集成鉴别器更准确地预测感知质量。