Deep neural networks, in particular convolutional neural networks, have
become highly effective tools for compressing images and solving inverse
problems including denoising, inpainting, and reconstruction from few and noisy
measurements. This success can be attributed in part to their ability to
represent and generate natural images well. Contrary to classical tools such as
wavelets, image-generating deep neural networks have a large number of
parameters---typically a multiple of their output dimension---and need to be
trained on large datasets. In this paper, we propose an untrained simple image
model, called the deep decoder, which is a deep neural network that can
generate natural images from very few weight parameters. The deep decoder has a
simple architecture with no convolutions and fewer weight parameters than the
output dimensionality. This underparameterization enables the deep decoder to
compress images into a concise set of network weights, which we show is on par
with wavelet-based thresholding. Further, underparameterization provides a
barrier to overfitting, allowing the deep decoder to have state-of-the-art
performance for denoising. The deep decoder is simple in the sense that each
layer has an identical structure that consists of only one upsampling unit,
pixel-wise linear combination of channels, ReLU activation, and channelwise
normalization. This simplicity makes the network amenable to theoretical
analysis, and it sheds light on the aspects of neural networks that enable them
to form effective signal representations.

本研究提出了一种未训练的简单图像模型 —— 深度解码器，它能够从非常少的权重参数生成自然图像，可以达到基于小波的阈值处理的效果，并且可以避免过拟合问题。此外，该模型简单且易于理论分析，为深度神经网络生成信号的有效表示提供了启示。