Symbolic regression aims to derive interpretable symbolic expressions from
data in order to better understand and interpret data. %which plays an
important role in knowledge discovery and interpretable machine learning.
In this study, a symbolic network called PruneSymNet is proposed for symbolic
regression. This is a novel neural network whose activation function consists
of common elementary functions and operators. The whole network is
differentiable and can be trained by gradient descent method. Each subnetwork
in the network corresponds to an expression, and our goal is to extract such
subnetworks to get the desired symbolic expression.
Therefore, a greedy pruning algorithm is proposed to prune the network into a
subnetwork while ensuring the accuracy of data fitting. The proposed greedy
pruning algorithm preserves the edge with the least loss in each pruning, but
greedy algorithm often can not get the optimal solution. In order to alleviate
this problem, we combine beam search during pruning to obtain multiple
candidate expressions each time, and finally select the expression with the
smallest loss as the final result. It was tested on the public data set and
compared with the current popular algorithms. The results showed that the
proposed algorithm had better accuracy.

提出了一种符号回归的符号网络 PruneSymNet，使用了贪婪修剪算法与波束搜索相结合的方法来获得最优解和多个候选表达式，并在公共数据集上进行了测试以及与流行算法的比较，结果表明该算法具有更好的准确性。

PruneSymNet：符号回归的符号神经网络和剪枝算法

PruneSymNet: A Symbolic Neural Network and Pruning Algorithm for  Symbolic Regression

We initiate the study of the inherent tradeoffs between the size of a neural
network and its robustness, as measured by its Lipschitz constant. We make a
precise conjecture that, for any Lipschitz activation function and for most
datasets, any two-layers neural network with $k$ neurons that perfectly fit the
data must have its Lipschitz constant larger (up to a constant) than
$\sqrt{n/k}$ where $n$ is the number of datapoints. In particular, this
conjecture implies that overparametrization is necessary for robustness, since
it means that one needs roughly one neuron per datapoint to ensure a
$O(1)$-Lipschitz network, while mere data fitting of $d$-dimensional data
requires only one neuron per $d$ datapoints. We prove a weaker version of this
conjecture when the Lipschitz constant is replaced by an upper bound on it
based on the spectral norm of the weight matrix. We also prove the conjecture
in the high-dimensional regime $n \approx d$ (which we also refer to as the
undercomplete case, since only $k \leq d$ is relevant here). Finally we prove
the conjecture for polynomial activation functions of degree $p$ when $n
\approx d^p$. We complement these findings with experimental evidence
supporting the conjecture.

研究显示神经网络的大小和唇氏常数之间存在固有的权衡，为保证唇氏恒定至少需要数据点数除以神经元个数的个数级别的神经元，过参数化（神经元数大于数据点数）是保证鲁棒性的必要条件，仅数据拟合仅需要 D 个数据点一个神经元。

两层神经网络的鲁棒性定律

A law of robustness for two-layers neural networks

An active learner is given a class of models, a large set of unlabeled
examples, and the ability to interactively query labels of a subset of these
examples; the goal of the learner is to learn a model in the class that fits
the data well.
Previous theoretical work has rigorously characterized label complexity of
active learning, but most of this work has focused on the PAC or the agnostic
PAC model. In this paper, we shift our attention to a more general setting --
maximum likelihood estimation. Provided certain conditions hold on the model
class, we provide a two-stage active learning algorithm for this problem. The
conditions we require are fairly general, and cover the widely popular class of
Generalized Linear Models, which in turn, include models for binary and
multi-class classification, regression, and conditional random fields.
We provide an upper bound on the label requirement of our algorithm, and a
lower bound that matches it up to lower order terms. Our analysis shows that
unlike binary classification in the realizable case, just a single extra round
of interaction is sufficient to achieve near-optimal performance in maximum
likelihood estimation. On the empirical side, the recent work in
~\cite{Zhang12} and~\cite{Zhang14} (on active linear and logistic regression)
shows the promise of this approach.

通过交互查询子集示例的标签，主动学习器在模型类和大量未标记示例的情况下学习适合数据的模型。本文提供了一个适用于极大似然估计的两阶段主动学习算法和标签需求的上限和下限分析，并表明这种方法在一些情况下可以实现近乎最优的性能。