Iteratively reweighted least square (IRLS) is a popular approach to solve
sparsity-enforcing regression problems in machine learning. State of the art
approaches are more efficient but typically rely on specific coordinate pruning
schemes. In this work, we show how a surprisingly simple reparametrization of
IRLS, coupled with a bilevel resolution (instead of an alternating scheme) is
able to achieve top performances on a wide range of sparsity (such as Lasso,
group Lasso and trace norm regularizations), regularization strength (including
hard constraints), and design matrices (ranging from correlated designs to
differential operators). Similarly to IRLS, our method only involves linear
systems resolutions, but in sharp contrast, corresponds to the minimization of
a smooth function. Despite being non-convex, we show that there is no spurious
minima and that saddle points are "ridable", so that there always exists a
descent direction. We thus advocate for the use of a BFGS quasi-Newton solver,
which makes our approach simple, robust and efficient. We perform a numerical
benchmark of the convergence speed of our algorithm against state of the art
solvers for Lasso, group Lasso, trace norm and linearly constrained problems.
These results highlight the versatility of our approach, removing the need to
use different solvers depending on the specificity of the ML problem under
study.

通过再参数化和双层解析度的相结合，我们提出了一种新的通过线性系统解决 Lasso 问题的方法，可以适用于各种疏松规则和设计矩阵，我们通过数值实验表明了该方法的高效性和鲁棒性。

稀疏正则化的平滑双层编程

Smooth Bilevel Programming for Sparse Regularization

Blind source separation (BSS) is a very popular technique to analyze
multichannel data. In this context, the data are modeled as the linear
combination of sources to be retrieved. For that purpose, standard BSS methods
all rely on some discrimination principle, whether it is statistical
independence or morphological diversity, to distinguish between the sources.
However, dealing with real-world data reveals that such assumptions are rarely
valid in practice: the signals of interest are more likely partially
correlated, which generally hampers the performances of standard BSS methods.
In this article, we introduce a novel sparsity-enforcing BSS method coined
Adaptive Morphological Component Analysis (AMCA), which is designed to retrieve
sparse and partially correlated sources. More precisely, it makes profit of an
adaptive re-weighting scheme to favor/penalize samples based on their level of
correlation. Extensive numerical experiments have been carried out which show
that the proposed method is robust to the partial correlation of sources while
standard BSS techniques fail. The AMCA algorithm is evaluated in the field of
astrophysics for the separation of physical components from microwave data.

介绍了一种名为 AMCA 的新型稀疏性 BSS 方法，旨在检索稀疏且部分相关的来源，并在宇宙物理学领域进行了广泛的数值实验以证明该算法的鲁棒性