This study comes as a timely response to mounting criticism of the
information bottleneck (IB) theory, injecting fresh perspectives to rectify
misconceptions and reaffirm its validity. Firstly, we introduce an auxiliary
function to reinterpret the maximal coding rate reduction method as a special
yet local optimal case of IB theory. Through this auxiliary function, we
clarify the paradox of decreasing mutual information during the application of
ReLU activation in deep learning (DL) networks. Secondly, we challenge the
doubts about IB theory's applicability by demonstrating its capacity to explain
the absence of a compression phase with linear activation functions in hidden
layers, when viewed through the lens of the auxiliary function. Lastly, by
taking a novel theoretical stance, we provide a new way to interpret the inner
organizations of DL networks by using IB theory, aligning them with recent
experimental evidence. Thus, this paper serves as an act of justice for IB
theory, potentially reinvigorating its standing and application in DL and other
fields such as communications and biomedical research.

本研究对信息瓶颈理论的应用进行了探讨，通过引入辅助函数和采取新的理论立场，解决了该理论在深度学习网络中的应用问题，并解释了 ReLU 激活下归零情形的信息瓶颈机制，从而提供了一种新的解读深度学习网络内部组织的方法。