We study the computational and sample complexity of learning a target
function $f_*:\mathbb{R}^d\to\mathbb{R}$ with additive structure, that is,
$f_*(x) = \frac{1}{\sqrt{M}}\sum_{m=1}^M f_m(\langle x, v_m\rangle)$, where
$f_1,f_2,...,f_M:\mathbb{R}\to\mathbb{R}$ are nonlinear link functions of
single-index models (ridge functions) with diverse and near-orthogonal index
features $\{v_m\}_{m=1}^M$, and the number of additive tasks $M$ grows with the
dimensionality $M\asymp d^\gamma$ for $\gamma\ge 0$. This problem setting is
motivated by the classical additive model literature, the recent representation
learning theory of two-layer neural network, and large-scale pretraining where
the model simultaneously acquires a large number of "skills" that are often
localized in distinct parts of the trained network. We prove that a large
subset of polynomial $f_*$ can be efficiently learned by gradient descent
training of a two-layer neural network, with a polynomial statistical and
computational complexity that depends on the number of tasks $M$ and the
information exponent of $f_m$, despite the unknown link function and $M$
growing with the dimensionality. We complement this learnability guarantee with
computational hardness result by establishing statistical query (SQ) lower
bounds for both the correlational SQ and full SQ algorithms.

研究具有加法结构的目标函数学习的计算和样本复杂性，证明了多项式 target function 可以通过梯度下降法训练两层神经网络高效地学习，同时建立了统计查询算法的边界。

学习多样特征之和：计算难度和基于梯度的高效训练用于冈脊组合

Learning sum of diverse features: computational hardness and efficient  gradient-based training for ridge combinations

We study the problem of learning mixtures of linear classifiers under
Gaussian covariates. Given sample access to a mixture of $r$ distributions on
$\mathbb{R}^n$ of the form $(\mathbf{x},y_{\ell})$, $\ell\in [r]$, where
$\mathbf{x}\sim\mathcal{N}(0,\mathbf{I}_n)$ and
$y_\ell=\mathrm{sign}(\langle\mathbf{v}_\ell,\mathbf{x}\rangle)$ for an unknown
unit vector $\mathbf{v}_\ell$, the goal is to learn the underlying distribution
in total variation distance. Our main result is a Statistical Query (SQ) lower
bound suggesting that known algorithms for this problem are essentially best
possible, even for the special case of uniform mixtures. In particular, we show
that the complexity of any SQ algorithm for the problem is
$n^{\mathrm{poly}(1/\Delta) \log(r)}$, where $\Delta$ is a lower bound on the
pairwise $\ell_2$-separation between the $\mathbf{v}_\ell$'s. The key technical
ingredient underlying our result is a new construction of spherical designs
that may be of independent interest.

学习高斯协变量下的线性分类器混合问题，主要结果是一个统计查询（SQ）下界暗示已知算法对于该问题基本上是最佳的，甚至对于均匀混合的特殊情况。

学习线性分类器混合模型的 SQ 下界

SQ Lower Bounds for Learning Mixtures of Linear Classifiers

We study the problem of high-dimensional linear regression in a robust model
where an $\epsilon$-fraction of the samples can be adversarially corrupted. We
focus on the fundamental setting where the covariates of the uncorrupted
samples are drawn from a Gaussian distribution $\mathcal{N}(0, \Sigma)$ on
$\mathbb{R}^d$. We give nearly tight upper bounds and computational lower
bounds for this problem. Specifically, our main contributions are as follows:
For the case that the covariance matrix is known to be the identity, we give
a sample near-optimal and computationally efficient algorithm that outputs a
candidate hypothesis vector $\widehat{\beta}$ which approximates the unknown
regression vector $\beta$ within $\ell_2$-norm $O(\epsilon \log(1/\epsilon)
\sigma)$, where $\sigma$ is the standard deviation of the random observation
noise. An error of $\Omega (\epsilon \sigma)$ is information-theoretically
necessary, even with infinite sample size. Prior work gave an algorithm for
this problem with sample complexity $\tilde{\Omega}(d^2/\epsilon^2)$ whose
error guarantee scales with the $\ell_2$-norm of $\beta$.
For the case of unknown covariance, we show that we can efficiently achieve
the same error guarantee as in the known covariance case using an additional
$\tilde{O}(d^2/\epsilon^2)$ unlabeled examples. On the other hand, an error of
$O(\epsilon \sigma)$ can be information-theoretically attained with
$O(d/\epsilon^2)$ samples. We prove a Statistical Query (SQ) lower bound
providing evidence that this quadratic tradeoff in the sample size is inherent.
More specifically, we show that any polynomial time SQ learning algorithm for
robust linear regression (in Huber's contamination model) with estimation
complexity $O(d^{2-c})$, where $c>0$ is an arbitrarily small constant, must
incur an error of $\Omega(\sqrt{\epsilon} \sigma)$.

研究了高维线性回归在对抗性污染下的稳健模型问题，并针对从高斯分布生成的未被修正的样本的基本情况给出了几乎最紧的上界和计算下界。