Deep unrolling, or unfolding, is an emerging learning-to-optimize method that
unrolls a truncated iterative algorithm in the layers of a trainable neural
network. However, the convergence guarantees and generalizability of the
unrolled networks are still open theoretical problems. To tackle these
problems, we provide deep unrolled architectures with a stochastic descent
nature by imposing descending constraints during training. The descending
constraints are forced layer by layer to ensure that each unrolled layer takes,
on average, a descent step toward the optimum during training. We theoretically
prove that the sequence constructed by the outputs of the unrolled layers is
then guaranteed to converge for unseen problems, assuming no distribution shift
between training and test problems. We also show that standard unrolling is
brittle to perturbations, and our imposed constraints provide the unrolled
networks with robustness to additive noise and perturbations. We numerically
assess unrolled architectures trained under the proposed constraints in two
different applications, including the sparse coding using learnable iterative
shrinkage and thresholding algorithm (LISTA) and image inpainting using
proximal generative flow (GLOW-Prox), and demonstrate the performance and
robustness benefits of the proposed method.

深度展开是一种新兴的学习优化方法，在可训练的神经网络中展开剪枝迭代算法的层，本文提出使用随机下降约束进行训练的深度展开结构，理论上证明了输出序列的收敛性和对未知问题的泛化能力，同时展示了对扰动和干扰具有鲁棒性的优势。