We consider the problem of robustly predicting as well as the best linear
combination of $d$ given functions in least squares regression, and variants of
this problem including constraints on the parameters of the linear combination.
For the ridge estimator and the ordinary least squares estimator, and their
variants, we provide new risk bounds of order $d/n$ without logarithmic factor
unlike some standard results, where $n$ is the size of the training data. We
also provide a new estimator with better deviations in the presence of
heavy-tailed noise. It is based on truncating differences of losses in a
min--max framework and satisfies a $d/n$ risk bound both in expectation and in
deviations. The key common surprising factor of these results is the absence of
exponential moment condition on the output distribution while achieving
exponential deviations. All risk bounds are obtained through a PAC-Bayesian
analysis on truncated differences of losses. Experimental results strongly back
up our truncated min--max estimator.

针对在普通最小二乘法回归中预测的偏差问题，我们提出了一个更好的估算方法 —— 基于截断误差差值的极小 - 极大框架，其期望和差距都为 d/n。

鲁棒线性最小二乘回归

Robust linear least squares regression

We derive sharp performance bounds for least squares regression with $L_1$
regularization from parameter estimation accuracy and feature selection quality
perspectives. The main result proved for $L_1$ regularization extends a similar
result in [Ann. Statist. 35 (2007) 2313--2351] for the Dantzig selector. It
gives an affirmative answer to an open question in [Ann. Statist. 35 (2007)
2358--2364]. Moreover, the result leads to an extended view of feature
selection that allows less restrictive conditions than some recent work. Based
on the theoretical insights, a novel two-stage $L_1$-regularization procedure
with selective penalization is analyzed. It is shown that if the target
parameter vector can be decomposed as the sum of a sparse parameter vector with
large coefficients and another less sparse vector with relatively small
coefficients, then the two-stage procedure can lead to improved performance.

通过参数估计准确性和特征选择质量两个角度，我们得出了最小二乘回归与 $L_1$ 正则化的性能边界。对于 $L_1$ 正则化得出的主要结果扩展了 Dantzig 选择器中 [Ann.Statist.35 (2007) 2313-2351] 类似的结果，并肯定回答了 [Ann.Statist.35 (2007) 2358-2364] 中的一个未解问题。此外，该结果还产生了一种更广泛的特征选择观点，该观点比一些最近的工作具有更少的约束条件。基于理论洞察力，我们分析了具有选择性处罚的新型两阶段 $L_1$ 正则化过程，并表明如果目标参数向量可以分解为具有大系数的稀疏参数向量和具有相对较小系数的其他较少稀疏向量的总和，则两阶段过程可以提高性能。

关于带有 L1 正则化的最小二乘回归的一些尖锐性能界

Some sharp performance bounds for least squares regression with $L_1$  regularization

We consider high-dimensional generalized linear models with Lipschitz loss
functions, and prove a nonasymptotic oracle inequality for the empirical risk
minimizer with Lasso penalty. The penalty is based on the coefficients in the
linear predictor, after normalization with the empirical norm. The examples
include logistic regression, density estimation and classification with hinge
loss. Least squares regression is also discussed.

研究了具有 Lipschitz 损失函数的高维广义线性模型，并证明了带有 Lasso 惩罚项的经验风险最小化算子的非渐进性 oracle 不等式。惩罚项是基于线性预测中的系数，在经验规范化后计算。研究包括逻辑回归、密度估计、带有 Hinge 损失的分类和最小二乘回归。