The classical sparse coding (SC) model represents visual stimuli as a linear
combination of a handful of learned basis functions that are Gabor-like when
trained on natural image data. However, the Gabor-like filters learned by
classical sparse coding far overpredict well-tuned simple cell receptive field
profiles observed empirically. While neurons fire sparsely, neuronal
populations are also organized in physical space by their sensitivity to
certain features. In V1, this organization is a smooth progression of
orientations along the cortical sheet. A number of subsequent models have
either discarded the sparse dictionary learning framework entirely or whose
updates have yet to take advantage of the surge in unrolled, neural dictionary
learning architectures. A key missing theme of these updates is a stronger
notion of \emph{structured sparsity}. We propose an autoencoder architecture
(WLSC) whose latent representations are implicitly, locally organized for
spectral clustering through a Laplacian quadratic form of a bipartite graph,
which generates a diverse set of artificial receptive fields that match primate
data in V1 as faithfully as recent contrastive frameworks like Local Low
Dimensionality, or LLD \citep{lld} that discard sparse dictionary learning. By
unifying sparse and smooth coding in models of the early visual cortex through
our autoencoder, we also show that our regularization can be interpreted as
early-stage specialization of receptive fields to certain classes of stimuli;
that is, we induce a weak clustering bias for later stages of cortex where
functional and spatial segregation (i.e. topography) are known to occur. The
results show an imperative for \emph{spatial regularization} of both the
receptive fields and firing rates to begin to describe feature disentanglement
in V1 and beyond.

在这篇论文中，我们提出了一种自动编码器架构（WLSC），其潜在表示通过二分图的拉普拉斯二次形式实现了隐式的、本地的谱聚类，生成了一组多样的人工感受野，与 V1 的灵长类动物数据具有相近的一致性，而且也展示了我们的正则化可以被解释为感受野对特定刺激类别的早期专门化，即我们为之后的皮质阶段引入了弱的聚类偏差，这是已知的功能性和空间分离（即拓扑）发生的地方。这些结果表明，在对 V1 及其之后的特征分离进行描述时，对感受野和放电率进行空间正则化至关重要。

使用展开网络进行聚类归纳偏差

Clustering Inductive Biases with Unrolled Networks

Sparse coding is a proven principle for learning compact representations of
images. However, sparse coding by itself often leads to very redundant
dictionaries. With images, this often takes the form of similar edge detectors
which are replicated many times at various positions, scales and orientations.
An immediate consequence of this observation is that the estimation of the
dictionary components is not statistically efficient. We propose a factored
model in which factors of variation (e.g. position, scale and orientation) are
untangled from the underlying Gabor-like filters. There is so much redundancy
in sparse codes for natural images that our model requires only a single
dictionary element (a Gabor-like edge detector) to outperform standard sparse
coding. Our model scales naturally to arbitrary-sized images while achieving
much greater statistical efficiency during learning. We validate this claim
with a number of experiments showing, in part, superior compression of
out-of-sample data using a sparse coding dictionary learned with only a single
image.

通过分离图像的变异因子，我们提出了一种基于 Gabor-like 滤波器的模型，在学习过程中达到更高的统计效率和更好的压缩性能，仅使用单个字典元素就可以优于标准稀疏编码，较好地解决了传统稀疏编码过度冗余的问题。