Encoding only the task-related information from the raw data, \ie,
disentangled representation learning, can greatly contribute to the robustness
and generalizability of models. Although significant advances have been made by
regularizing the information in representations with information theory, two
major challenges remain: 1) the representation compression inevitably leads to
performance drop; 2) the disentanglement constraints on representations are in
complicated optimization. To these issues, we introduce Bayesian networks with
transmitted information to formulate the interaction among input and
representations during disentanglement. Building upon this framework, we
propose \textbf{DisTIB} (\textbf{T}ransmitted \textbf{I}nformation
\textbf{B}ottleneck for \textbf{Dis}entangled representation learning), a novel
objective that navigates the balance between information compression and
preservation. We employ variational inference to derive a tractable estimation
for DisTIB. This estimation can be simply optimized via standard gradient
descent with a reparameterization trick. Moreover, we theoretically prove that
DisTIB can achieve optimal disentanglement, underscoring its superior efficacy.
To solidify our claims, we conduct extensive experiments on various downstream
tasks to demonstrate the appealing efficacy of DisTIB and validate our
theoretical analyses.

通过传递信息的贝叶斯网络，我们引入了 DisTIB（传递信息瓶颈）作为一种导航信息压缩和保留之间平衡的新目标，通过可变推理和重参数化技巧获得了可优化的估计，实验证明其能够实现最优的解缠效果。