In the realm of neural network models, the perpetual challenge remains in
retaining task-relevant information while effectively discarding redundant data
during propagation. In this paper, we introduce IB-AdCSCNet, a deep learning
model grounded in information bottleneck theory. IB-AdCSCNet seamlessly
integrates the information bottleneck trade-off strategy into deep networks by
dynamically adjusting the trade-off hyperparameter $\lambda$ through gradient
descent, updating it within the FISTA(Fast Iterative Shrinkage-Thresholding
Algorithm ) framework. By optimizing the compressive excitation loss function
induced by the information bottleneck principle, IB-AdCSCNet achieves an
optimal balance between compression and fitting at a global level,
approximating the globally optimal representation feature. This information
bottleneck trade-off strategy driven by downstream tasks not only helps to
learn effective features of the data, but also improves the generalization of
the model. This study's contribution lies in presenting a model with consistent
performance and offering a fresh perspective on merging deep learning with
sparse representation theory, grounded in the information bottleneck concept.
Experimental results on CIFAR-10 and CIFAR-100 datasets demonstrate that
IB-AdCSCNet not only matches the performance of deep residual convolutional
networks but also outperforms them when handling corrupted data. Through the
inference of the IB trade-off, the model's robustness is notably enhanced.

本研究介绍了一种基于信息瓶颈理论的深度学习模型 IB-AdCSCNet，通过在梯度下降中动态调整交换参数 λ，优化压缩激发损失函数，实现压缩和拟合的最佳平衡。该模型不仅提供了一种一致性表现的模型，还融合了稀疏表示理论与深度学习的前沿视角。实验结果表明，IB-AdCSCNet 在处理受损数据时不仅能与深度残差卷积网络相媲美，而且还能表现更佳，并通过 IB 交换的推断显著增强了模型的稳健性。