Deep networks are increasingly being applied to problems involving image
synthesis, e.g., generating images from textual descriptions and reconstructing
an input image from a compact representation. Supervised training of
image-synthesis networks typically uses a pixel-wise loss (PL) to indicate the
mismatch between a generated image and its corresponding target image. We
propose instead to use a loss function that is better calibrated to human
perceptual judgments of image quality: the multiscale structural-similarity
score (MS-SSIM). Because MS-SSIM is differentiable, it is easily incorporated
into gradient-descent learning. We compare the consequences of using MS-SSIM
versus PL loss on training deterministic and stochastic autoencoders. For three
different architectures, we collected human judgments of the quality of image
reconstructions. Observers reliably prefer images synthesized by
MS-SSIM-optimized models over those synthesized by PL-optimized models, for two
distinct PL measures ($\ell_1$ and $\ell_2$ distances). We also explore the
effect of training objective on image encoding and analyze conditions under
which perceptually-optimized representations yield better performance on image
classification. Finally, we demonstrate the superiority of
perceptually-optimized networks for super-resolution imaging. Just as computer
vision has advanced through the use of convolutional architectures that mimic
the structure of the mammalian visual system, we argue that significant
additional advances can be made in modeling images through the use of training
objectives that are well aligned to characteristics of human perception.

通过使用 better calibrated to human perceptual judgments of image quality: the multiscale structural-similarity score (MS-SSIM) 进行训练而不是使用 pixel-wise loss (PL)，提出了更好的 perceptually-optimized methods，已被证明在三种不同的自动编码器中表现更好，可以在图像分类和超分辨率成像方面为计算机视觉带来极大的潜力。