In this work, we propose Cell Variational Information Bottleneck Network
(cellVIB), a convolutional neural network using information bottleneck
mechanism, which can be combined with the latest feedforward network
architecture in an end-to-end training method. Our Cell Variational Information
Bottleneck Network is constructed by stacking VIB cells, which generate feature
maps with uncertainty. As layers going deeper, the regularization effect will
gradually increase, instead of directly adding excessive regular constraints to
the output layer of the model as in Deep VIB. Under each VIB cell, the
feedforward process learns an independent mean term and an standard deviation
term, and predicts the Gaussian distribution based on them. The feedback
process is based on reparameterization trick for effective training. This work
performs an extensive analysis on MNIST dataset to verify the effectiveness of
each VIB cells, and provides an insightful analysis on how the VIB cells affect
mutual information. Experiments conducted on CIFAR-10 also prove that our
cellVIB is robust against noisy labels during training and against corrupted
images during testing. Then, we validate our method on PACS dataset, whose
results show that the VIB cells can significantly improve the generalization
performance of the basic model. Finally, in a more complex representation
learning task, face recognition, our network structure has also achieved very
competitive results.

本研究提出了一种使用信息瓶颈机制的卷积神经网络 —— 细胞变分信息瓶颈网络（cellVIB），它可以与最新的前馈网络结构相结合，在端到端训练方法中使用。我们的细胞变分信息瓶颈网络通过堆叠 VIB 细胞来构建，可以生成具有不确定性的特征映射。以 MNIST 数据集为例，对每个 VIB 细胞进行广泛分析以验证其有效性，并对 VIB 细胞如何影响互信息进行深入分析。在 CIFAR-10 上进行的实验也证明了我们的 cellVIB 对训练过程中的噪声标签和测试中的损坏图像具有鲁棒性。然后，我们在 PACS 数据集上验证了我们的方法，结果表明 VIB 细胞可以显著提高基本模型的泛化性能。最后，在更复杂的表示学习任务 —— 人脸识别中，我们的网络结构也取得了非常有竞争力的结果。