Online statistical inference facilitates real-time analysis of sequentially
collected data, making it different from traditional methods that rely on
static datasets. This paper introduces a novel approach to online inference in
high-dimensional generalized linear models, where we update regression
coefficient estimates and their standard errors upon each new data arrival. In
contrast to existing methods that either require full dataset access or
large-dimensional summary statistics storage, our method operates in a
single-pass mode, significantly reducing both time and space complexity. The
core of our methodological innovation lies in an adaptive stochastic gradient
descent algorithm tailored for dynamic objective functions, coupled with a
novel online debiasing procedure. This allows us to maintain low-dimensional
summary statistics while effectively controlling optimization errors introduced
by the dynamically changing loss functions. We demonstrate that our method,
termed the Approximated Debiased Lasso (ADL), not only mitigates the need for
the bounded individual probability condition but also significantly improves
numerical performance. Numerical experiments demonstrate that the proposed ADL
method consistently exhibits robust performance across various covariance
matrix structures.

在线统计推断使得实时分析顺序采集的数据成为可能，本文引入了一种针对高维广义线性模型的在线推断新方法，通过在每次新增数据到达时更新回归系数估计和其标准误差，与现有方法相比，该方法以单次传递模式运行，大大降低了时间和空间复杂度。方法的核心创新在于针对动态目标函数设计的自适应随机梯度下降算法，结合了一种新型的在线去偏过程，能够在有效控制由动态变化的损失函数引入的优化误差的同时，保持低维度的摘要统计量。我们的方法，即近似去偏套索（ADL），不仅减轻了有界个别概率条件的需求，而且显著提高了数值性能。数值实验证明了所提出的 ADL 方法在各种协方差矩阵结构下一致表现出鲁棒性。

高维广义线性模型中基于流数据的自适应无偏 SGD

Adaptive debiased SGD in high-dimensional GLMs with steaming data

In high-dimensional generalized linear models, it is crucial to identify a
sparse model that adequately accounts for response variation. Although the best
subset section has been widely regarded as the Holy Grail of problems of this
type, achieving either computational efficiency or statistical guarantees is
challenging. In this article, we intend to surmount this obstacle by utilizing
a fast algorithm to select the best subset with high certainty. We proposed and
illustrated an algorithm for best subset recovery in regularity conditions.
Under mild conditions, the computational complexity of our algorithm scales
polynomially with sample size and dimension. In addition to demonstrating the
statistical properties of our method, extensive numerical experiments reveal
that it outperforms existing methods for variable selection and coefficient
estimation. The runtime analysis shows that our implementation achieves
approximately a fourfold speedup compared to popular variable selection
toolkits like glmnet and ncvreg.

在高维广义线性模型中，通过使用快速算法选择最佳子集，以高确定性解决了多变量选择的问题并证明了统计性质。

广义线性模型中的最佳子集选择：一种基于剪接技术的快速且一致的算法

Best-Subset Selection in Generalized Linear Models: A Fast and  Consistent Algorithm via Splicing Technique

Empirical Risk Minimization (ERM) algorithms are widely used in a variety of
estimation and prediction tasks in signal-processing and machine learning
applications. Despite their popularity, a theory that explains their
statistical properties in modern regimes where both the number of measurements
and the number of unknown parameters is large is only recently emerging. In
this paper, we characterize for the first time the fundamental limits on the
statistical accuracy of convex ERM for inference in high-dimensional
generalized linear models. For a stylized setting with Gaussian features and
problem dimensions that grow large at a proportional rate, we start with sharp
performance characterizations and then derive tight lower bounds on the
estimation and prediction error that hold over a wide class of loss functions
and for any value of the regularization parameter. Our precise analysis has
several attributes. First, it leads to a recipe for optimally tuning the loss
function and the regularization parameter. Second, it allows to precisely
quantify the sub-optimality of popular heuristic choices: for instance, we show
that optimally-tuned least-squares is (perhaps surprisingly) approximately
optimal for standard logistic data, but the sub-optimality gap grows
drastically as the signal strength increases. Third, we use the bounds to
precisely assess the merits of ridge-regularization as a function of the
over-parameterization ratio. Notably, our bounds are expressed in terms of the
Fisher Information of random variables that are simple functions of the data
distribution, thus making ties to corresponding bounds in classical statistics.

本文首次表征凸形 ERM 在高维广义线性模型推断中的基本统计精度界限，推导出任意损失函数和正则化参数值的紧凑下界，并精确评价了损失函数和正则化参数值的优化调整。