Deep neural networks trained in an end-to-end manner are proven to be
efficient in a wide range of machine learning tasks. However, there is one
drawback of end-to-end learning: The learned features and information are
implicitly represented in neural network parameters, which cannot be used as
regularities, concepts or knowledge to explicitly represent the data
probability distribution. To resolve this issue, we propose in this paper a new
machine learning theory, which defines in mathematics what are regularities.
Briefly, regularities are concise representations of the non-random features,
or 'non-randomness' in the data probability distribution. Combining with
information theory, we claim that regularities can also be regarded as a small
amount of information encoding a large amount of information. Our theory is
based on spiking functions. That is, if a function can react to, or spike on
specific data samples more frequently than random noise inputs, we say that
such a function discovers non-randomness from the data distribution, and
encodes the non-randomness into regularities. Our theory also discusses
applying multiple spiking functions to the same data distribution. In this
process, we claim that the 'best' regularities, or the optimal spiking
functions, are those who can capture the largest amount of information from the
data distribution, and then encode the captured information in the most concise
way. Theorems and hypotheses are provided to describe in mathematics what are
'best' regularities and optimal spiking functions. Finally, we propose a
machine learning approach, which can potentially obtain the optimal spiking
functions regarding the given dataset in practice.

提出了一种新的机器学习理论，定义了数学上的规律性，通过结合信息理论，将规律性看作大量信息中编码的少量信息，并讨论了将多个脉冲函数应用于相同数据分布的过程，并提出了一种机器学习方法，可以在实践中获得给定数据集的最佳脉冲函数。

利用脉冲函数从数据中学习规律：一种理论

Learning Regularities from Data using Spiking Functions: A Theory

Neural models for the various flavours of morphological inflection tasks have
proven to be extremely accurate given ample labeled data -- data that may be
slow and costly to obtain. In this work we aim to overcome this annotation
bottleneck by bootstrapping labeled data from a seed as little as {\em five}
labeled paradigms, accompanied by a large bulk of unlabeled text. Our approach
exploits different kinds of regularities in morphological systems in a
two-phased setup, where word tagging based on {\em analogies} is followed by
word pairing based on {\em distances}. We experiment with the Paradigm Cell
Filling Problem over eight typologically different languages, and find that, in
languages with relatively simple morphology, orthographic regularities on their
own allow inflection models to achieve respectable accuracy. Combined
orthographic and semantic regularities alleviate difficulties with particularly
complex morpho-phonological systems. Our results suggest that hand-crafting
many tagged examples might be an unnecessary effort. However, more work is
needed in order to address rarely used forms.

该研究使用少量标记的数据引导标注词形变化，利用不同类型的规律，通过类比词标记和距离对词配对，证明了手工标记许多示例可能是不必要的。

形态学屈折的最小监督

Minimal Supervision for Morphological Inflection

We consider hundreds of thousands of individual economic transactions to ask:
how predictable are consumers in their merchant visitation patterns? Our
results suggest that, in the long-run, much of our seemingly elective activity
is actually highly predictable. Notwithstanding a wide range of individual
preferences, shoppers share regularities in how they visit merchant locations
over time. Yet while aggregate behavior is largely predictable, the
interleaving of shopping events introduces important stochastic elements at
short time scales. These short- and long-scale patterns suggest a theoretical
upper bound on predictability, and describe the accuracy of a Markov model in
predicting a person's next location. We incorporate population-level transition
probabilities in the predictive models, and find that in many cases these
improve accuracy. While our results point to the elusiveness of precise
predictions about where a person will go next, they suggest the existence, at
large time-scales, of regularities across the population.

研究商家消费数据，发现尽管个人消费习惯各有不同，但在时间上有一定的规律性，而人群行为更容易预测，尤其是基于马尔科夫模型的预测。