Convolutional neural networks are among the most successful architectures in
deep learning with this success at least partially attributable to the efficacy
of spatial invariance as an inductive bias. Locally connected layers, which
differ from convolutional layers only in their lack of spatial invariance,
usually perform poorly in practice. However, these observations still leave
open the possibility that some degree of relaxation of spatial invariance may
yield a better inductive bias than either convolution or local connectivity. To
test this hypothesis, we design a method to relax the spatial invariance of a
network layer in a controlled manner; we create a \textit{low-rank} locally
connected layer, where the filter bank applied at each position is constructed
as a linear combination of basis set of filter banks with spatially varying
combining weights. By varying the number of basis filter banks, we can control
the degree of relaxation of spatial invariance. In experiments with small
convolutional networks, we find that relaxing spatial invariance improves
classification accuracy over both convolution and locally connected layers
across MNIST, CIFAR-10, and CelebA datasets, thus suggesting that spatial
invariance may be an overly restrictive prior.

为了测试假设认为部分放松空间不变性能够提供一个更好的归纳偏置，本文设计了一种方法来以可控的方式放松神经网络层的空间不变性。实验中发现，相比于卷积层和局部连接层，放松空间不变性能够在 MNIST，CIFAR-10 和 CelebA 数据集上改善分类精度，因此暗示空间不变性可能是一种过度限制的先验。

通过低秩局部连接重新审视空间不变性

Revisiting Spatial Invariance with Low-Rank Local Connectivity

In recent years, Spiking Neural Networks (SNNs) have demonstrated great
successes in completing various Machine Learning tasks. We introduce a method
for learning image features by \textit{locally connected layers} in SNNs using
spike-timing-dependent plasticity (STDP) rule. In our approach, sub-networks
compete via competitive inhibitory interactions to learn features from
different locations of the input space. These \textit{Locally-Connected SNNs}
(LC-SNNs) manifest key topological features of the spatial interaction of
biological neurons. We explore biologically inspired n-gram classification
approach allowing parallel processing over various patches of the the image
space. We report the classification accuracy of simple two-layer LC-SNNs on two
image datasets, which match the state-of-art performance and are the first
results to date. LC-SNNs have the advantage of fast convergence to a dataset
representation, and they require fewer learnable parameters than other SNN
approaches with unsupervised learning. Robustness tests demonstrate that
LC-SNNs exhibit graceful degradation of performance despite the random deletion
of large amounts of synapses and neurons.

使用局部连接层和时序性突触可塑性规则的脉冲神经网络 (LC-SNNs) 能够优化图像分类任务，其通过接受来自不同位置输入的竞争性抑制相互交互的方式实现生物神经元的拓扑特征，并在两个图像数据集上表现出先进水平的分类精度并展示了较强的稳健性能。