A framework is presented for unsupervised learning of representations based
on infomax principle for large-scale neural populations. We use an asymptotic
approximation to the Shannon's mutual information for a large neural population
to demonstrate that a good initial approximation to the global
information-theoretic optimum can be obtained by a hierarchical infomax method.
Starting from the initial solution, an efficient algorithm based on gradient
descent of the final objective function is proposed to learn representations
from the input datasets, and the method works for complete, overcomplete, and
undercomplete bases. As confirmed by numerical experiments, our method is
robust and highly efficient for extracting salient features from input
datasets. Compared with the main existing methods, our algorithm has a distinct
advantage in both the training speed and the robustness of unsupervised
representation learning. Furthermore, the proposed method is easily extended to
the supervised or unsupervised model for training deep structure networks.

提出了一种基于 infomax 原理的框架，可实现对大规模神经群体进行无监督学习。该方法使用基于渐近的方法来对大规模神经群体的信息论下限进行计算，通过渐进性的不断往全局信息论最优解靠近的过程，可以获得一个很好的初始值。基于该初始值，提出了一个高效的算法来从输入数据集中学习特征表示，并且该方法适用于完备、过完备和欠完备情况下的基函数。和现有的方法相比，该算法在无监督表示学习的训练速度和鲁棒性方面都具有明显的优势，而且还可以轻松地扩展到用于训练深层结构网络的监督或无监督模型。

神经群体 Infomax 快速稳健非监督学习的信息论框架

An Information-Theoretic Framework for Fast and Robust Unsupervised  Learning via Neural Population Infomax

A new algorithm is proposed for a) unsupervised learning of sparse
representations from subsampled measurements and b) estimating the parameters
required for linearly reconstructing signals from the sparse codes. We verify
that the new algorithm performs efficient data compression on par with the
recent method of compressive sampling. Further, we demonstrate that the
algorithm performs robustly when stacked in several stages or when applied in
undercomplete or overcomplete situations. The new algorithm can explain how
neural populations in the brain that receive subsampled input through fiber
bottlenecks are able to form coherent response properties.

提出了一种新算法，用于从子采样测量中无监督地学习稀疏表示并估计用于从稀疏编码线性重构信号的参数，进一步证明该算法在数据压缩方面效率与压缩采样的最近方法相当，同时演示在堆叠多层或在欠完备或过完备情况下应用时该算法的鲁棒性，该算法能够解释大脑中接收通过纤维瓶颈的子采样输入的神经群体如何形成连贯的响应特性。