Human object recognition exhibits remarkable resilience in cluttered and
dynamic visual environments. In contrast, despite their unparalleled
performance across numerous visual tasks, Deep Neural Networks (DNNs) remain
far less robust than humans, showing, for example, a surprising susceptibility
to adversarial attacks involving image perturbations that are (almost)
imperceptible to humans. Human object recognition likely owes its robustness,
in part, to the increasingly resilient representations that emerge along the
hierarchy of the ventral visual cortex. Here we show that DNNs, when guided by
neural representations from a hierarchical sequence of regions in the human
ventral visual stream, display increasing robustness to adversarial attacks.
These neural-guided models also exhibit a gradual shift towards more human-like
decision-making patterns and develop hierarchically smoother decision surfaces.
Importantly, the resulting representational spaces differ in important ways
from those produced by conventional smoothing methods, suggesting that such
neural-guidance may provide previously unexplored robustness solutions. Our
findings support the gradual emergence of human robustness along the ventral
visual hierarchy and suggest that the key to DNN robustness may lie in
increasing emulation of the human brain.

DNNs guided by neural representations from the human ventral visual stream display increasing robustness to adversarial attacks, develop more human-like decision-making patterns, and suggest new robustness solutions by emulating the human brain.

利用人类腹侧视觉通路提高神经网络的稳健性

Leveraging the Human Ventral Visual Stream to Improve Neural Network  Robustness

Human ability to recognize complex visual patterns arises through
transformations performed by successive areas in the ventral visual cortex.
Deep neural networks trained end-to-end for object recognition approach human
capabilities, and offer the best descriptions to date of neural responses in
the late stages of the hierarchy. But these networks provide a poor account of
the early stages, compared to traditional hand-engineered models, or models
optimized for coding efficiency or prediction. Moreover, the gradient
backpropagation used in end-to-end learning is generally considered to be
biologically implausible. Here, we overcome both of these limitations by
developing a bottom-up self-supervised training methodology that operates
independently on successive layers. Specifically, we maximize feature
similarity between pairs of locally-deformed natural image patches, while
decorrelating features across patches sampled from other images. Crucially, the
deformation amplitudes are adjusted proportionally to receptive field sizes in
each layer, thus matching the task complexity to the capacity at each stage of
processing. In comparison with architecture-matched versions of previous
models, we demonstrate that our layerwise complexity-matched learning (LCL)
formulation produces a two-stage model (LCL-V2) that is better aligned with
selectivity properties and neural activity in primate area V2. We demonstrate
that the complexity-matched learning paradigm is critical for the emergence of
the improved biological alignment. Finally, when the two-stage model is used as
a fixed front-end for a deep network trained to perform object recognition, the
resultant model (LCL-V2Net) is significantly better than standard end-to-end
self-supervised, supervised, and adversarially-trained models in terms of
generalization to out-of-distribution tasks and alignment with human behavior.

通过在连续层上独立进行自我监督训练的自底向上方法，我们使两阶段的模型更好地与灵长类动物的 V2 区域神经活动一致，提高了对于识别对象和与人类行为相一致的一致性。