We consider the capacity of \emph{treelike committee machines} (TCM) neural
networks. Relying on Random Duality Theory (RDT), \cite{Stojnictcmspnncaprdt23}
recently introduced a generic framework for their capacity analysis. An upgrade
based on the so-called \emph{partially lifted} RDT (pl RDT) was then presented
in \cite{Stojnictcmspnncapliftedrdt23}. Both lines of work focused on the
networks with the most typical, \emph{sign}, activations. Here, on the other
hand, we focus on networks with other, more general, types of activations and
show that the frameworks of
\cite{Stojnictcmspnncaprdt23,Stojnictcmspnncapliftedrdt23} are sufficiently
powerful to enable handling of such scenarios as well. In addition to the
standard \emph{linear} activations, we uncover that particularly convenient
results can be obtained for two very commonly used activations, namely, the
\emph{quadratic} and \emph{rectified linear unit (ReLU)} ones. In more concrete
terms, for each of these activations, we obtain both the RDT and pl RDT based
memory capacities upper bound characterization for \emph{any} given (even)
number of the hidden layer neurons, $d$. In the process, we also uncover the
following two, rather remarkable, facts: 1) contrary to the common wisdom, both
sets of results show that the bounding capacity decreases for large $d$ (the
width of the hidden layer) while converging to a constant value; and 2) the
maximum bounding capacity is achieved for the networks with precisely
\textbf{\emph{two}} hidden layer neurons! Moreover, the large $d$ converging
values are observed to be in excellent agrement with the statistical physics
replica theory based predictions.

根据随机二重性理论 (RDT)，本研究利用部分整体化 RDT 的框架分析了具有不同激活函数的树状委员会机器 (TCM) 神经网络的容量，特别关注于线性、二次和修正线性单元 (ReLU) 激活函数，并发现隐藏层神经元个数为两个时容量最大。