Algorithm unfolding or unrolling is the technique of constructing a deep
neural network (DNN) from an iterative algorithm. Unrolled DNNs often provide
better interpretability and superior empirical performance over standard DNNs
in signal estimation tasks. An important theoretical question, which has only
recently received attention, is the development of generalization error bounds
for unrolled DNNs. These bounds deliver theoretical and practical insights into
the performance of a DNN on empirical datasets that are distinct from, but
sampled from, the probability density generating the DNN training data. In this
paper, we develop novel generalization error bounds for a class of unrolled
DNNs that are informed by a compound Gaussian prior. These compound Gaussian
networks have been shown to outperform comparative standard and unfolded deep
neural networks in compressive sensing and tomographic imaging problems. The
generalization error bound is formulated by bounding the Rademacher complexity
of the class of compound Gaussian network estimates with Dudley's integral.
Under realistic conditions, we show that, at worst, the generalization error
scales $\mathcal{O}(n\sqrt{\ln(n)})$ in the signal dimension and
$\mathcal{O}(($Network Size$)^{3/2})$ in network size.

通过边界限制 Rademacher 复杂度，我们为一类来自复合高斯先验的展开深度神经网络开发了新颖的泛化误差边界，可在信号维度和网络规模方面进行评估。

关于深度复合高斯神经网络的泛化界限

On Generalization Bounds for Deep Compound Gaussian Neural Networks

Solving linear inverse problems plays a crucial role in numerous
applications. Algorithm unfolding based, model-aware data-driven approaches
have gained significant attention for effectively addressing these problems.
Learned iterative soft-thresholding algorithm (LISTA) and alternating direction
method of multipliers compressive sensing network (ADMM-CSNet) are two widely
used such approaches, based on ISTA and ADMM algorithms, respectively. In this
work, we study optimization guarantees, i.e., achieving near-zero training loss
with the increase in the number of learning epochs, for finite-layer unfolded
networks such as LISTA and ADMM-CSNet with smooth soft-thresholding in an
over-parameterized (OP) regime. We achieve this by leveraging a modified
version of the Polyak-Lojasiewicz, denoted PL$^*$, condition. Satisfying the
PL$^*$ condition within a specific region of the loss landscape ensures the
existence of a global minimum and exponential convergence from initialization
using gradient descent based methods. Hence, we provide conditions, in terms of
the network width and the number of training samples, on these unfolded
networks for the PL$^*$ condition to hold. We achieve this by deriving the
Hessian spectral norm of these networks. Additionally, we show that the
threshold on the number of training samples increases with the increase in the
network width. Furthermore, we compare the threshold on training samples of
unfolded networks with that of a standard fully-connected feed-forward network
(FFNN) with smooth soft-thresholding non-linearity. We prove that unfolded
networks have a higher threshold value than FFNN. Consequently, one can expect
a better expected error for unfolded networks than FFNN.

通过研究无穷展开网络（如 LISTA 和 ADMM-CSNet）中的平滑软阈值函数的优化保证，本文证明了存在于损失函数空间特定区域内满足 PL$^*$ 条件，从而保证全局最小值的存在和使用梯度下降方法的指数级收敛。此外，本文还比较了无穷展开网络与标准的全连接前向网络（FFNN）的训练样本数量阈值，并证明无穷展开网络具有更高的阈值，因此预期无穷展开网络的期望误差将优于 FFNN。