Backpropagation-optimized artificial neural networks, while precise, lack
robustness, leading to unforeseen behaviors that affect their safety.
Biological neural systems do solve some of these issues already. Thus,
understanding the biological mechanisms of robustness is an important step
towards building trustworthy and safe systems. Unlike artificial models,
biological neurons adjust connectivity based on neighboring cell activity.
Robustness in neural representations is hypothesized to correlate with the
smoothness of the encoding manifold. Recent work suggests power law covariance
spectra, which were observed studying the primary visual cortex of mice, to be
indicative of a balanced trade-off between accuracy and robustness in
representations. Here, we show that unsupervised local learning models with
winner takes all dynamics learn such power law representations, providing
upcoming studies a mechanistic model with that characteristic. Our research
aims to understand the interplay between geometry, spectral properties,
robustness, and expressivity in neural representations. Hence, we study the
link between representation smoothness and spectrum by using weight, Jacobian
and spectral regularization while assessing performance and adversarial
robustness. Our work serves as a foundation for future research into the
mechanisms underlying power law spectra and optimally smooth encodings in both
biological and artificial systems. The insights gained may elucidate the
mechanisms that realize robust neural networks in mammalian brains and inform
the development of more stable and reliable artificial systems.

通过权重、Jacobian 和谱规范化评估性能和对抗性鲁棒性，研究了表示平滑性和谱之间的关联，为未来探究动物和人工系统中的幂律谱和最优平滑编码机制奠定基础，从而更好地理解实现哺乳动物大脑中鲁棒神经网络和开发更加稳定可靠的人工系统的机制。