We introduce a new computational framework for estimating parameters in
generalized generalized linear models (GGLM), a class of models that extends
the popular generalized linear models (GLM) to account for dependencies among
observations in spatio-temporal data. The proposed approach uses a monotone
operator-based variational inequality method to overcome non-convexity in
parameter estimation and provide guarantees for parameter recovery. The results
can be applied to GLM and GGLM, focusing on spatio-temporal models. We also
present online instance-based bounds using martingale concentrations
inequalities. Finally, we demonstrate the performance of the algorithm using
numerical simulations and a real data example for wildfire incidents.

提出一个计算新框架以估算广义广义线性模型 (GGLM) 中的参数，该模型扩展了流行的广义线性模型 (GLM) 以解释时空数据中的观测依赖性。该方法使用单调算子为基础的变分不等式方法来克服参数估计中的非凸性并提供参数恢复的保证，适用于 GLM 和 GGLM，重点关注时空模型，并使用数值模拟和真实数据实例演示算法的性能。

广义广义线性模型：凸估计和在线界限

Generalized generalized linear models: Convex estimation and online  bounds

Logistic regression is commonly used for modeling dichotomous outcomes. In
the classical setting, where the number of observations is much larger than the
number of parameters, properties of the maximum likelihood estimator in
logistic regression are well understood. Recently, Sur and Candes have studied
logistic regression in the high-dimensional regime, where the number of
observations and parameters are comparable, and show, among other things, that
the maximum likelihood estimator is biased. In the high-dimensional regime the
underlying parameter vector is often structured (sparse, block-sparse,
finite-alphabet, etc.) and so in this paper we study regularized logistic
regression (RLR), where a convex regularizer that encourages the desired
structure is added to the negative of the log-likelihood function. An advantage
of RLR is that it allows parameter recovery even for instances where the
(unconstrained) maximum likelihood estimate does not exist. We provide a
precise analysis of the performance of RLR via the solution of a system of six
nonlinear equations, through which any performance metric of interest (mean,
mean-squared error, probability of support recovery, etc.) can be explicitly
computed. Our results generalize those of Sur and Candes and we provide a
detailed study for the cases of $\ell_2^2$-RLR and sparse
($\ell_1$-regularized) logistic regression. In both cases, we obtain explicit
expressions for various performance metrics and can find the values of the
regularizer parameter that optimizes the desired performance. The theory is
validated by extensive numerical simulations across a range of parameter values
and problem instances.

本文研究了高维情况下正则化逻辑回归（RLR），其中加入了鼓励所需结构的凸正则项。通过求解一组非线性方程组，我们提供了 RLR 性能的精确分析，并获得了各种性能度量的显式表达式。我们进行了广泛的数值模拟，并在各种参数值和问题实例中验证了理论。

正则化对高维逻辑回归的影响

The Impact of Regularization on High-dimensional Logistic Regression

In this paper, we consider parameter recovery for non-overlapping
convolutional neural networks (CNNs) with multiple kernels. We show that when
the inputs follow Gaussian distribution and the sample size is sufficiently
large, the squared loss of such CNNs is $\mathit{~locally~strongly~convex}$ in
a basin of attraction near the global optima for most popular activation
functions, like ReLU, Leaky ReLU, Squared ReLU, Sigmoid and Tanh. The required
sample complexity is proportional to the dimension of the input and polynomial
in the number of kernels and a condition number of the parameters. We also show
that tensor methods are able to initialize the parameters to the local strong
convex region. Hence, for most smooth activations, gradient descent following
tensor initialization is guaranteed to converge to the global optimal with time
that is linear in input dimension, logarithmic in precision and polynomial in
other factors. To the best of our knowledge, this is the first work that
provides recovery guarantees for CNNs with multiple kernels under polynomial
sample and computational complexities.

本研究考虑具有多个内核的不重叠卷积神经网络的参数恢复，当输入服从高斯分布且样本量足够大时，对于大多数流行的激活函数，如 ReLU，Leaky ReLU，Squared ReLU，Sigmoid 和 Tanh，我们展示了该 CNN 的平方损失在全局最优附近的吸引盆中是局部强凸的，所需样本复杂度与输入维度成比例且多项式内核数量和参数的条件数，同时我们还展示了张量方法能够将参数初始化为局部强凸的区域，因此，对于大多数光滑的激活函数，张量初始化后的梯度下降保证在输入维度、精度对数和其他因素上是多项式数量级的时间内收敛到全局最优解。据我们所知，这是第一份提供具有多个内核的 CNN 的恢复保证的工作，其样本复杂度和计算复杂度都是多项式的。