Recently, Ye et al. (Mathematical Programming 2023) designed an algorithm for
solving a specific class of bilevel programs with an emphasis on applications
related to hyperparameter selection, utilizing the difference of convex
algorithm based on the value function approach reformulation. The proposed
algorithm is particularly powerful when the lower level problem is fully convex
, such as a support vector machine model or a least absolute shrinkage and
selection operator model. In this paper, to suit more applications related to
machine learning and statistics, we substantially weaken the underlying
assumption from lower level full convexity to weak convexity. Accordingly, we
propose a new reformulation using Moreau envelope of the lower level problem
and demonstrate that this reformulation is a difference of weakly convex
program. Subsequently, we develop a sequentially convergent algorithm for
solving this difference of weakly convex program. To evaluate the effectiveness
of our approach, we conduct numerical experiments on the bilevel hyperparameter
selection problem from elastic net, sparse group lasso, and RBF kernel support
vector machine models.

该研究设计了一种利用凸差算法基于价值函数法的重要双层规划算法， 以解决与超参数选择相关的应用问题，提出了在满足弱凸度假设的情况下根据下层问题 Moreau envelope 的差分弱凸规划改进措施，并验证了其有效性。

基于莫罗包络的弱凸差分重构和双层规划算法

Moreau Envelope Based Difference-of-weakly-Convex Reformulation and  Algorithm for Bilevel Programs

Bilevel programs are optimization problems where some variables are solutions
to optimization problems themselves, and they arise in a variety of control
applications, including: control of vehicle traffic networks, inverse
reinforcement learning and inverse optimization, and robust control for
human-automation systems. This paper develops a duality-based approach to
solving bilevel programs where the lower level problem is convex. Our approach
is to use partial dualization to construct a new dual function that is
differentiable, unlike the Lagrangian dual that is only directionally
differentiable. We use our dual to define a duality-based reformulation of
bilevel programs, prove equivalence of our reformulation with the original
bilevel program, and then introduce regularization to ensure constraint
qualification holds. These technical results about our new dual and regularized
duality-based reformulation are used to provide theoretical justification for
an algorithm we construct for solving bilevel programs with a convex lower
level, and we conclude by demonstrating the efficacy of our algorithm by
solving two practical instances of bilevel programs.

本文提出了一种基于对偶理论的方法来解决具有凸低层问题的双层规划问题，并且引入了一种优化方法以保证约束条件成立。作者通过提出的构建新对偶函数方法获得技术性结果，并最终通过求解两个实例来演示该算法的有效性。