Higher order artificial neurons whose outputs are computed by applying an
activation function to a higher order multinomial function of the inputs have
been considered in the past, but did not gain acceptance due to the extra
parameters and computational cost. However, higher order neurons have
significantly greater learning capabilities since the decision boundaries of
higher order neurons can be complex surfaces instead of just hyperplanes. The
boundary of a single quadratic neuron can be a general hyper-quadric surface
allowing it to learn many nonlinearly separable datasets. Since quadratic forms
can be represented by symmetric matrices, only $\frac{n(n+1)}{2}$ additional
parameters are needed instead of $n^2$. A quadratic Logistic regression model
is first presented. Solutions to the XOR problem with a single quadratic neuron
are considered. The complete vectorized equations for both forward and backward
propagation in feedforward networks composed of quadratic neurons are derived.
A reduced parameter quadratic neural network model with just $ n $ additional
parameters per neuron that provides a compromise between learning ability and
computational cost is presented. Comparison on benchmark classification
datasets are used to demonstrate that a final layer of quadratic neurons
enables networks to achieve higher accuracy with significantly fewer hidden
layer neurons. In particular this paper shows that any dataset composed of $C$
bounded clusters can be separated with only a single layer of $C$ quadratic
neurons.

通过引入高阶人工神经元，本论文介绍了具有增强的学习能力的二次神经元、解决 XOR 问题的方法以及应用于前向神经网络的全矢量化方程。与隐藏层神经元数量相比，仅利用一层二次神经元可以使网络在更少的隐藏层神经元的情况下实现更高的准确性，从而在基准分类数据集上取得了显著的成果。