Biologically, the brain does not rely on a single type of neuron that
universally functions in all aspects. Instead, it acts as a sophisticated
designer of task-based neurons. In this study, we address the following
question: since the human brain is a task-based neuron user, can the artificial
network design go from the task-based architecture design to the task-based
neuron design? Since methodologically there are no one-size-fits-all neurons,
given the same structure, task-based neurons can enhance the feature
representation ability relative to the existing universal neurons due to the
intrinsic inductive bias for the task. Specifically, we propose a two-step
framework for prototyping task-based neurons. First, symbolic regression is
used to identify optimal formulas that fit input data by utilizing base
functions such as logarithmic, trigonometric, and exponential functions. We
introduce vectorized symbolic regression that stacks all variables in a vector
and regularizes each input variable to perform the same computation, which can
expedite the regression speed, facilitate parallel computation, and avoid
overfitting. Second, we parameterize the acquired elementary formula to make
parameters learnable, which serves as the aggregation function of the neuron.
The activation functions such as ReLU and the sigmoidal functions remain the
same because they have proven to be good. Empirically, experimental results on
synthetic data, classic benchmarks, and real-world applications show that the
proposed task-based neuron design is not only feasible but also delivers
competitive performance over other state-of-the-art models.

人工网络设计可以从基于任务的架构设计到基于任务的神经元设计，采用符号回归和参数化等方法，可以实现竞争性能提升。