In various domains within imaging and data science, particularly when
addressing tasks modeled utilizing the variational regularization approach,
manually configuring regularization parameters presents a formidable challenge.
The difficulty intensifies when employing regularizers involving a large number
of hyperparameters. To overcome this challenge, bilevel learning is employed to
learn suitable hyperparameters. However, due to the use of numerical solvers,
the exact gradient with respect to the hyperparameters is unattainable,
necessitating the use of methods relying on approximate gradients.
State-of-the-art inexact methods a priori select a decreasing summable sequence
of the required accuracy and only assure convergence given a sufficiently small
fixed step size. Despite this, challenges persist in determining the Lipschitz
constant of the hypergradient and identifying an appropriate fixed step size.
Conversely, computing exact function values is not feasible, impeding the use
of line search. In this work, we introduce a provably convergent inexact
backtracking line search involving inexact function evaluations and
hypergradients. We show convergence to a stationary point of the loss with
respect to hyperparameters. Additionally, we propose an algorithm to determine
the required accuracy dynamically. Our numerical experiments demonstrate the
efficiency and feasibility of our approach for hyperparameter estimation in
variational regularization problems, alongside its robustness in terms of the
initial accuracy and step size choices.

使用变分正则化方法，通过双层学习来学习合适的超参数，同时提出了能够收敛到损失函数超参数的稳定点的近似回溯线搜索算法，并在变分正则化问题的超参数估计中展示了其有效性和可行性。

动态双层学习与不精确线搜索

Dynamic Bilevel Learning with Inexact Line Search

Variational regularization is commonly used to solve linear inverse problems,
and involves augmenting a data fidelity by a regularizer. The regularizer is
used to promote a priori information, and is weighted by a regularization
parameter. Selection of an appropriate regularization parameter is critical,
with various choices leading to very different reconstructions. Existing
strategies such as the discrepancy principle and L-curve can be used to
determine a suitable parameter value, but in recent years a supervised machine
learning approach called bilevel learning has been employed. Bilevel learning
is a powerful framework to determine optimal parameters, and involves solving a
nested optimisation problem. While previous strategies enjoy various
theoretical results, the well-posedness of bilevel learning in this setting is
still a developing field. One necessary property is positivity of the
determined regularization parameter. In this work, we provide a new condition
that better characterises positivity of optimal regularization parameters than
the existing theory. Numerical results verify and explore this new condition
for both small and large dimensional problems.

利用变分正则化方法求解线性反问题时，采用超参数调节正则化项，通过强化先验信息来提高求解结果，核心问题在于如何选择一个合适的正则化参数。本文提出了一个新的条件来更好地描述正则化参数的正性，并验证和探索了这个新条件，同时也探究了这个新条件在小和大维问题上的应用。