Deep neural networks trained in an end-to-end manner are proven to be
efficient in a wide range of machine learning tasks. However, there is one
drawback of end-to-end learning: The learned features and information are
implicitly represented in neural network parameters, which cannot be used as
regularities, concepts or knowledge to explicitly represent the data
probability distribution. To resolve this issue, we propose in this paper a new
machine learning theory, which defines in mathematics what are regularities.
Briefly, regularities are concise representations of the non-random features,
or 'non-randomness' in the data probability distribution. Combining with
information theory, we claim that regularities can also be regarded as a small
amount of information encoding a large amount of information. Our theory is
based on spiking functions. That is, if a function can react to, or spike on
specific data samples more frequently than random noise inputs, we say that
such a function discovers non-randomness from the data distribution, and
encodes the non-randomness into regularities. Our theory also discusses
applying multiple spiking functions to the same data distribution. In this
process, we claim that the 'best' regularities, or the optimal spiking
functions, are those who can capture the largest amount of information from the
data distribution, and then encode the captured information in the most concise
way. Theorems and hypotheses are provided to describe in mathematics what are
'best' regularities and optimal spiking functions. Finally, we propose a
machine learning approach, which can potentially obtain the optimal spiking
functions regarding the given dataset in practice.

提出了一种新的机器学习理论，定义了数学上的规律性，通过结合信息理论，将规律性看作大量信息中编码的少量信息，并讨论了将多个脉冲函数应用于相同数据分布的过程，并提出了一种机器学习方法，可以在实践中获得给定数据集的最佳脉冲函数。