Recently, Optimization-Derived Learning (ODL) has attracted attention from
learning and vision areas, which designs learning models from the perspective
of optimization. However, previous ODL approaches regard the training and
hyper-training procedures as two separated stages, meaning that the
hyper-training variables have to be fixed during the training process, and thus
it is also impossible to simultaneously obtain the convergence of training and
hyper-training variables. In this work, we design a Generalized
Krasnoselskii-Mann (GKM) scheme based on fixed-point iterations as our
fundamental ODL module, which unifies existing ODL methods as special cases.
Under the GKM scheme, a Bilevel Meta Optimization (BMO) algorithmic framework
is constructed to solve the optimal training and hyper-training variables
together. We rigorously prove the essential joint convergence of the
fixed-point iteration for training and the process of optimizing
hyper-parameters for hyper-training, both on the approximation quality, and on
the stationary analysis. Experiments demonstrate the efficiency of BMO with
competitive performance on sparse coding and real-world applications such as
image deconvolution and rain streak removal.

本文研究基于固定点迭代的广义 Krasnoselskii-Mann（GKM）方案，设计了一个双层元优化算法框架（BMO），以同时解决最佳训练和超级训练变量。通过实验表明，BMO 在稀疏编码和实际应用（例如图像去卷积和雨脉冲去除）中具有高效的竞争性能。