Theoretical properties of bilevel problems are well studied when the
lower-level problem is strongly convex. In this work, we focus on bilevel
optimization problems without the strong-convexity assumption. In these cases,
we first show that the common local optimality measures such as KKT condition
or regularization can lead to undesired consequences. Then, we aim to identify
the mildest conditions that make bilevel problems tractable. We identify two
classes of growth conditions on the lower-level objective that leads to
continuity. Under these assumptions, we show that the local optimality of the
bilevel problem can be defined via the Goldstein stationarity condition of the
hyper-objective. We then propose the Inexact Gradient-Free Method (IGFM) to
solve the bilevel problem, using an approximate zeroth order oracle that is of
independent interest. Our non-asymptotic analysis demonstrates that the
proposed method can find a $(\delta, \varepsilon)$ Goldstein stationary point
for bilevel problems with a zeroth order oracle complexity that is polynomial
in $d, 1/\delta$ and $1/\varepsilon$.

本文关注于无强凸性假设的双层优化问题中局部最优性条件的探讨，提出了两类下层目标的增长条件从而推导出 Goldstein 稳定性条件，并引入了 Inexact Gradient-Free Method 方法进行求解。

关于非下层强凸性的双层优化问题

On Bilevel Optimization without Lower-level Strong Convexity

Langevin diffusion is a commonly used tool for sampling from a given
distribution. In this work, we establish that when the target density $p^*$ is
such that $\log p^*$ is $L$ smooth and $m$ strongly convex, discrete Langevin
diffusion produces a distribution $p$ with $KL(p||p^*)\leq \epsilon$ in
$\tilde{O}(\frac{d}{\epsilon})$ steps, where $d$ is the dimension of the sample
space. We also study the convergence rate when the strong-convexity assumption
is absent. By considering the Langevin diffusion as a gradient flow in the
space of probability distributions, we obtain an elegant analysis that applies
to the stronger property of convergence in KL-divergence and gives a
conceptually simpler proof of the best-known convergence results in weaker
metrics.

研究 Langevin 扩散在采样、KL - 散度、强凸性、收敛速率等方面的应用，证明在目标密度是 L 光滑且 m 强凸的情况下，该扩散可以在几步内收敛于目标分布，同时揭示了在强凸性假设缺失时的收敛速率。

Langevin MCMC 在 KL 散度下的收敛性

Convergence of Langevin MCMC in KL-divergence

We consider optimizing a function smooth convex function $f$ that is the
average of a set of differentiable functions $f_i$, under the assumption
considered by Solodov [1998] and Tseng [1998] that the norm of each gradient
$f_i'$ is bounded by a linear function of the norm of the average gradient
$f'$. We show that under these assumptions the basic stochastic gradient method
with a sufficiently-small constant step-size has an $O(1/k)$ convergence rate,
and has a linear convergence rate if $g$ is strongly-convex.

本文考虑优化一个平滑凸函数，该函数是一组可微函数的平均数，在每个梯度的范数受到平均梯度范数的线性约束的假设下，证明了基本的随机梯度方法具有 O (1/k) 的收敛速度，并且在强凸条件下具有线性收敛速度。