Few neural architectures lend themselves to provable learning with gradient
based methods. One popular model is the single-index model, in which labels are
produced by composing an unknown linear projection with a possibly unknown
scalar link function. Learning this model with SGD is relatively
well-understood, whereby the so-called information exponent of the link
function governs a polynomial sample complexity rate. However, extending this
analysis to deeper or more complicated architectures remains challenging.
In this work, we consider single index learning in the setting of symmetric
neural networks. Under analytic assumptions on the activation and maximum
degree assumptions on the link function, we prove that gradient flow recovers
the hidden planted direction, represented as a finitely supported vector in the
feature space of power sum polynomials. We characterize a notion of information
exponent adapted to our setting that controls the efficiency of learning.

在对称神经网络的设置下，通过对激活函数进行分析和对连接函数进行最大度数的假设，我们证明了梯度流可以恢复隐藏的预设方向，该方向在幂和多项式特征空间中表示为一个有限支持的向量，并刻画了适应我们设置的信息指数概念来控制学习的效率。

对称单指标学习

Symmetric Single Index Learning

We study the problem of signal estimation from non-linear observations when
the signal belongs to a low-dimensional set buried in a high-dimensional space.
A rough heuristic often used in practice postulates that non-linear
observations may be treated as noisy linear observations, and thus the signal
may be estimated using the generalized Lasso. This is appealing because of the
abundance of efficient, specialized solvers for this program. Just as noise may
be diminished by projecting onto the lower dimensional space, the error from
modeling non-linear observations with linear observations will be greatly
reduced when using the signal structure in the reconstruction. We allow general
signal structure, only assuming that the signal belongs to some set K in R^n.
We consider the single-index model of non-linearity. Our theory allows the
non-linearity to be discontinuous, not one-to-one and even unknown. We assume a
random Gaussian model for the measurement matrix, but allow the rows to have an
unknown covariance matrix. As special cases of our results, we recover
near-optimal theory for noisy linear observations, and also give the first
theoretical accuracy guarantee for 1-bit compressed sensing with unknown
covariance matrix of the measurement vectors.

研究了非线性观测信号估计问题，当信号属于高维空间中的低维集合时，可以使用广义 Lasso 方法，针对非线性观测信号进行噪声线性观察建模，通过信号重建结构降低误差，允许信号具有不连续、多义和未知的非线性特征，并允许测量矩阵的行具有未知的协方差矩阵。

非线性观测下的广义 Lasso

The generalized Lasso with non-linear observations

Let $(\bX, Y)$ be a random pair taking values in $\mathbb R^p \times \mathbb
R$. In the so-called single-index model, one has $Y=f^{\star}(\theta^{\star
T}\bX)+\bW$, where $f^{\star}$ is an unknown univariate measurable function,
$\theta^{\star}$ is an unknown vector in $\mathbb R^d$, and $W$ denotes a
random noise satisfying $\mathbb E[\bW|\bX]=0$. The single-index model is known
to offer a flexible way to model a variety of high-dimensional real-world
phenomena. However, despite its relative simplicity, this dimension reduction
scheme is faced with severe complications as soon as the underlying dimension
becomes larger than the number of observations ("$p$ larger than $n$"
paradigm). To circumvent this difficulty, we consider the single-index model
estimation problem from a sparsity perspective using a PAC-Bayesian approach.
On the theoretical side, we offer a sharp oracle inequality, which is more
powerful than the best known oracle inequalities for other common procedures of
single-index recovery. The proposed method is implemented by means of the
reversible jump Markov chain Monte Carlo technique and its performance is
compared with that of standard procedures.

本文从稀疏性角度，采用 PAC-Bayesian 方法对单指数模型估计问题进行了研究，在理论上提出了一个更强大的预测界，比单指数恢复的其他常见程序的最佳预测界更强大，利用可逆跳跃马尔可夫链蒙特卡洛技术实现了所提出的方法并将其与标准程序进行了比较。