In \cite{Hop82}, Hopfield introduced a \emph{Hebbian} learning rule based
neural network model and suggested how it can efficiently operate as an
associative memory. Studying random binary patterns, he also uncovered that, if
a small fraction of errors is tolerated in the stored patterns retrieval, the
capacity of the network (maximal number of memorized patterns, $m$) scales
linearly with each pattern's size, $n$. Moreover, he famously predicted
$\alpha_c=\lim_{n\rightarrow\infty}\frac{m}{n}\approx 0.14$. We study this very
same scenario with two famous pattern's basins of attraction:
\textbf{\emph{(i)}} The AGS one from \cite{AmiGutSom85}; and
\textbf{\emph{(ii)}} The NLT one from
\cite{Newman88,Louk94,Louk94a,Louk97,Tal98}. Relying on the \emph{fully lifted
random duality theory} (fl RDT) from \cite{Stojnicflrdt23}, we obtain the
following explicit capacity characterizations on the first level of lifting:
\begin{equation}
\alpha_c^{(AGS,1)} = \left ( \max_{\delta\in \left ( 0,\frac{1}{2}\right )
}\frac{1-2\delta}{\sqrt{2} \mbox{erfinv} \left ( 1-2\delta\right )} -
\frac{2}{\sqrt{2\pi}} e^{-\left ( \mbox{erfinv}\left ( 1-2\delta \right )\right
)^2}\right )^2 \approx \mathbf{0.137906} \end{equation}
\begin{equation}
\alpha_c^{(NLT,1)} = \frac{\mbox{erf}(x)^2}{2x^2}-1+\mbox{erf}(x)^2 \approx
\mathbf{0.129490}, \quad 1-\mbox{erf}(x)^2-
\frac{2\mbox{erf}(x)e^{-x^2}}{\sqrt{\pi}x}+\frac{2e^{-2x^2}}{\pi}=0.
\end{equation}
A substantial numerical work gives on the second level of lifting
$\alpha_c^{(AGS,2)} \approx \mathbf{0.138186}$ and $\alpha_c^{(NLT,2)} \approx
\mathbf{0.12979}$, effectively uncovering a remarkably fast lifting
convergence. Moreover, the obtained AGS characterizations exactly match the
replica symmetry based ones of \cite{AmiGutSom85} and the corresponding
symmetry breaking ones of \cite{SteKuh94}.

文中介绍了一种基于赫布学习规则的神经网络模型，并研究了模式检索和容量特性，使用完全升级随机对偶理论得到精确的容量特性，并展示了快速升级收敛的结果。

赫布规范 - 霍普菲尔德网络关联记忆的容量

Capacity of the Hebbian-Hopfield network associative memory

Dense Hopfield networks are known for their feature to prototype transition
and adversarial robustness. However, previous theoretical studies have been
mostly concerned with their storage capacity. We bridge this gap by studying
the phase diagram of p-body Hopfield networks in the teacher-student setting of
an unsupervised learning problem, uncovering ferromagnetic phases reminiscent
of the prototype and feature learning regimes. On the Nishimori line, we find
the critical size of the training set necessary for efficient pattern
retrieval. Interestingly, we find that that the paramagnetic to ferromagnetic
transition of the teacher-student setting coincides with the paramagnetic to
spin-glass transition of the direct model, i.e. with random patterns. Outside
of the Nishimori line, we investigate the learning performance in relation to
the inference temperature and dataset noise. Moreover, we show that using a
larger p for the student than the teacher gives the student an extensive
tolerance to noise. We then derive a closed-form expression measuring the
adversarial robustness of such a student at zero temperature, corroborating the
positive correlation between number of parameters and robustness observed in
large neural networks. We also use our model to clarify why the prototype phase
of modern Hopfield networks is adversarially robust.

Dense Hopfield 网络在特征到原型的转换和对抗性鲁棒性方面具有特色。本研究通过研究无监督学习问题中教师 - 学生设置的 p-body Hopfield 网络的相图，在模式检索方面揭示了类似于原型学习和特征学习区域的铁磁相。我们发现在 Nishimori 线上，需要训练集的临界大小才能实现高效的模式检索。有趣的是，我们发现教师 - 学生设置的顺磁到铁磁的转变与直接模型（即随机模式）的顺磁到自旋玻璃的转变相一致。在 Nishimori 线之外，我们研究了学习性能与推断温度和数据集噪声的关系。此外，我们证明学生的 p 比教师更大可以给学生以对噪声的广泛容忍性。然后，我们导出了一个在零温下衡量这种学生对抗性鲁棒性的闭合形式表达式，证实了大型神经网络中观察到的参数数量与鲁棒性之间的正相关性。我们还使用我们的模型阐明了现代 Hopfield 网络的原型相为何具有对抗性鲁棒性。