Information Bottleneck (IB) is a widely used framework that enables the
extraction of information related to a target random variable from a source
random variable. In the objective function, IB controls the trade-off between
data compression and predictiveness through the Lagrange multiplier $\beta$.
Traditionally, to find the trade-off to be learned, IB requires a search for
$\beta$ through multiple training cycles, which is computationally expensive.
In this study, we introduce Flexible Variational Information Bottleneck (FVIB),
an innovative framework for classification task that can obtain optimal models
for all values of $\beta$ with single, computationally efficient training. We
theoretically demonstrate that across all values of reasonable $\beta$, FVIB
can simultaneously maximize an approximation of the objective function for
Variational Information Bottleneck (VIB), the conventional IB method. Then we
empirically show that FVIB can learn the VIB objective as effectively as VIB.
Furthermore, in terms of calibration performance, FVIB outperforms other IB and
calibration methods by enabling continuous optimization of $\beta$. Our codes
are available at this https URL

本研究提出了一种创新的分类任务框架，称为 Flexible Variational Information Bottleneck (FVIB)，通过单一、高效的训练可以获取在所有 $eta$ 值上的最优模型，并且在校准性能方面优于其他信息瓶颈和校准方法。