Applied statisticians use sequential regression procedures to produce a
ranking of explanatory variables and, in settings of low correlations between
variables and strong true effect sizes, expect that variables at the very top
of this ranking are truly relevant to the response. In a regime of certain
sparsity levels, however, three examples of sequential procedures--forward
stepwise, the lasso, and least angle regression--are shown to include the first
spurious variable unexpectedly early. We derive a rigorous, sharp prediction of
the rank of the first spurious variable for these three procedures,
demonstrating that the first spurious variable occurs earlier and earlier as
the regression coefficients become denser. This counterintuitive phenomenon
persists for statistically independent Gaussian random designs and an
arbitrarily large magnitude of the true effects. We gain a better understanding
of the phenomenon by identifying the underlying cause and then leverage the
insights to introduce a simple visualization tool termed the double-ranking
diagram to improve on sequential methods. As a byproduct of these findings, we
obtain the first provable result certifying the exact equivalence between the
lasso and least angle regression in the early stages of solution paths beyond
orthogonal designs. This equivalence can seamlessly carry over many important
model selection results concerning the lasso to least angle regression.

研究表明，在稀疏水平的情况下，顺序回归程序中的第一个虚假变量出现得越来越早。这种反直觉的现象是独立高斯随机设计和真实效应任意大的情况下具有持久性的。通过识别其潜在原因，我们介绍了一个简单的可视化工具 - double-ranking diagram，以改进序列方法，并获得了第一个证明套索和最小角回归在正交设计之外的解决路径的早期阶段的确切等价的证明。