We propose a novel objective function for learning robust deep
representations of data based on information theory. Data is projected into a
feature-vector space such that the mutual information of all subsets of
features relative to the supervising signal is maximized. This objective
function gives rise to robust representations by conserving available
information relative to supervision in the face of noisy or unavailable
features. Although the objective function is not directly tractable, we are
able to derive a surrogate objective function. Minimizing this surrogate loss
encourages features to be non-redundant and conditionally independent relative
to the supervising signal. To evaluate the quality of obtained solutions, we
have performed a set of preliminary experiments that show promising results.

本文提出一种基于信息论的学习鲁棒性深度表示的新颖目标函数，通过将数据投影到特征矢量空间，最大化相对于监督信号的所有特征子集的互信息，得到鲁棒表示，其对噪声或不可用特征的信息保留能力较强，我们利用一种替代目标函数最小化的方式实现此目标函数并进行实验验证。