This work addresses meta-learning (ML) by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units results in sparse representations from each model layer, as the units are organized into blocks where only one unit generates a non-zero output. The main operating principle of the introduced units rely on stochastic principles, as the network performs posterior sampling over competing units to select the winner. Therefore, the proposed networks are explicitly designed to extract input data representations of sparse stochastic nature, as opposed to the currently standard deterministic representation paradigm. Our approach produces state-of-the-art predictive accuracy on few-shot image classification and regression experiments, as well as reduced predictive error on an active learning setting; these improvements come with an immensely reduced computational cost.

通过引入随机局部胜者通吃（LWTA）激活的深层网络，本文解决了元学习问题，该类型的网络单元通过将单元组织成块，其中只有一个单元生成非零输出，从而导致来自每个模型层的稀疏表示。所提出的网络明确设计为提取稀疏随机性质的输入数据表示，我们的方法在少样本图像分类和回归实验中产生了最先进的预测准确性，并在主动学习设置中减少了预测误差，并大幅降低了计算成本。

基于线性竞争单元的随机深度网络用于模型无关元学习