Popular artificial neural networks (ANN) optimize parameters for
unidirectional value propagation, assuming some guessed parametrization type
like Multi-Layer Perceptron (MLP) or Kolmogorov-Arnold Network (KAN). In
contrast, for biological neurons e.g. "it is not uncommon for axonal
propagation of action potentials to happen in both directions" \cite{axon} -
suggesting they are optimized to continuously operate in multidirectional way.
Additionally, statistical dependencies a single neuron could model is not just
(expected) value dependence, but entire joint distributions including also
higher moments. Such agnostic joint distribution neuron would allow for
multidirectional propagation (of distributions or values) e.g. $\rho(x|y,z)$ or
$\rho(y,z|x)$ by substituting to $\rho(x,y,z)$ and normalizing. There will be
discussed Hierarchical Correlation Reconstruction (HCR) for such neuron model:
assuming $\rho(x,y,z)=\sum_{ijk} a_{ijk} f_i(x) f_j(y) f_k(z)$ type
parametrization of joint distribution with polynomial basis $f_i$, which allows
for flexible, inexpensive processing including nonlinearities, direct model
estimation and update, trained through standard backpropagation or novel ways
for such structure up to tensor decomposition. Using only pairwise
(input-output) dependencies, its expected value prediction becomes KAN-like
with trained activation functions as polynomials, can be extended by adding
higher order dependencies through included products - in conscious
interpretable way, allowing for multidirectional propagation of both values and
probability densities.

人工神经网络对参数进行优化，而生物神经元是多向传播的，可以建模整体联合分布，本文讨论了一种基于多项式基函数的分布神经元的层次相关重构，并通过标准反向传播或新方法进行训练，实现多向传播的数值和概率密度。

基于分层相关重建的生物启发式联合分布神经元，支持多方向神经网络

Biology-inspired joint distribution neurons based on Hierarchical  Correlation Reconstruction allowing for multidirectional neural networks

In this era of artificial intelligence, deep neural networks like
Convolutional Neural Networks (CNNs) have emerged as front-runners, often
surpassing human capabilities. These deep networks are often perceived as the
panacea for all challenges. Unfortunately, a common downside of these networks
is their ''black-box'' character, which does not necessarily mirror the
operation of biological neural systems. Some even have millions/billions of
learnable (tunable) parameters, and their training demands extensive data and
time.
Here, we integrate the principles of biological neurons in certain layer(s)
of CNNs. Specifically, we explore the use of neuro-science-inspired
computational models of the Lateral Geniculate Nucleus (LGN) and simple cells
of the primary visual cortex. By leveraging such models, we aim to extract
image features to use as input to CNNs, hoping to enhance training efficiency
and achieve better accuracy. We aspire to enable shallow networks with a
Push-Pull Combination of Receptive Fields (PP-CORF) model of simple cells as
the foundation layer of CNNs to enhance their learning process and performance.
To achieve this, we propose a two-tower CNN, one shallow tower and the other as
ResNet 18. Rather than extracting the features blindly, it seeks to mimic how
the brain perceives and extracts features. The proposed system exhibits a
noticeable improvement in the performance (on an average of $5\%-10\%$) on
CIFAR-10, CIFAR-100, and ImageNet-100 datasets compared to ResNet-18. We also
check the efficiency of only the Push-Pull tower of the network.

通过结合生物神经元原理，使用基于神经科学的计算模型来增强训练效率和提高准确性，以推动组合感受野模型作为卷积神经网络的基础层，从而在 CIFAR-10、CIFAR-100 和 ImageNet-100 数据集上实现性能显著提升（平均提升 5%-10%）

利用生物神经元属性的卷积神经网络

Convolutional Neural Networks Exploiting Attributes of Biological  Neurons

The vulnerability to adversarial perturbations is a major flaw of Deep Neural
Networks (DNNs) that raises question about their reliability when in real-world
scenarios. On the other hand, human perception, which DNNs are supposed to
emulate, is highly robust to such perturbations, indicating that there may be
certain features of the human perception that make it robust but are not
represented in the current class of DNNs. One such feature is that the activity
of biological neurons is correlated and the structure of this correlation tends
to be rather rigid over long spans of times, even if it hampers performance and
learning. We hypothesize that integrating such constraints on the activations
of a DNN would improve its adversarial robustness, and, to test this
hypothesis, we have developed the Self-Consistent Activation (SCA) layer, which
comprises of neurons whose activations are consistent with each other, as they
conform to a fixed, but learned, covariability pattern. When evaluated on image
and sound recognition tasks, the models with a SCA layer achieved high
accuracy, and exhibited significantly greater robustness than multi-layer
perceptron models to state-of-the-art Auto-PGD adversarial attacks
\textit{without being trained on adversarially perturbed data

通过引入自相关激活层，该研究以人类感知的特性为依据，改善了深度神经网络对敌对扰动的抵抗能力，从而提高了图像和声音识别任务的准确率并显著降低了对最新的自动 PGD 敌对攻击的脆弱性。