We develop a theory of finite-dimensional polyhedral subsets over the
Wasserstein space and optimization of functionals over them via first-order
methods. Our main application is to the problem of mean-field variational
inference, which seeks to approximate a distribution $\pi$ over $\mathbb{R}^d$
by a product measure $\pi^\star$. When $\pi$ is strongly log-concave and
log-smooth, we provide (1) approximation rates certifying that $\pi^\star$ is
close to the minimizer $\pi^\star_\diamond$ of the KL divergence over a
\emph{polyhedral} set $\mathcal{P}_\diamond$, and (2) an algorithm for
minimizing $\text{KL}(\cdot\|\pi)$ over $\mathcal{P}_\diamond$ with accelerated
complexity $O(\sqrt \kappa \log(\kappa d/\varepsilon^2))$, where $\kappa$ is
the condition number of $\pi$.

有限维多面体子集、Wasserstein 空间、均场变分推断、最小化算法和 KL 散度。

通过在 Wasserstein 空间中的多面体优化实现均场变分推断的算法

Algorithms for mean-field variational inference via polyhedral  optimization in the Wasserstein space

In this work we present a novel optimization strategy for image
reconstruction tasks under analysis-based image regularization, which promotes
sparse and/or low-rank solutions in some learned transform domain. We
parameterize such regularizers using potential functions that correspond to
weighted extensions of the $\ell_p^p$-vector and $\mathcal{S}_p^p$
Schatten-matrix quasi-norms with $0 < p \le 1$. Our proposed minimization
strategy extends the Iteratively Reweighted Least Squares (IRLS) method,
typically used for synthesis-based $\ell_p$ and $\mathcal{S}_p$ norm and
analysis-based $\ell_1$ and nuclear norm regularization. We prove that under
mild conditions our minimization algorithm converges linearly to a stationary
point, and we provide an upper bound for its convergence rate. Further, to
select the parameters of the regularizers that deliver the best results for the
problem at hand, we propose to learn them from training data by formulating the
supervised learning process as a stochastic bilevel optimization problem. We
show that thanks to the convergence guarantees of our proposed minimization
strategy, such optimization can be successfully performed with a
memory-efficient implicit back-propagation scheme. We implement our learned
IRLS variants as recurrent networks and assess their performance on the
challenging image reconstruction tasks of non-blind deblurring,
super-resolution and demosaicking. The comparisons against other existing
learned reconstruction approaches demonstrate that our overall method is very
competitive and in many cases outperforms existing unrolled networks, whose
number of parameters is orders of magnitude higher than in our case.

该研究提出了一种新颖的优化策略，用于分析图像正则化下的图像重建任务，推动在一些学习转换域中稀疏和 / 或低秩解，并通过学习网络实现了较高性能。

迭代重新加权最小二乘网络：求解反向成像问题的收敛保证

Iterative Reweighted Least Squares Networks With Convergence Guarantees  for Solving Inverse Imaging Problems

Minimizing a function over an intersection of convex sets is an important
task in optimization that is often much more challenging than minimizing it
over each individual constraint set. While traditional methods such as
Frank-Wolfe (FW) or proximal gradient descent assume access to a linear or
quadratic oracle on the intersection, splitting techniques take advantage of
the structure of each sets, and only require access to the oracle on the
individual constraints. In this work, we develop and analyze the Frank-Wolfe
Augmented Lagrangian (FW-AL) algorithm, a method for minimizing a smooth
function over convex compact sets related by a "linear consistency" constraint
that only requires access to a linear minimization oracle over the individual
constraints. It is based on the Augmented Lagrangian Method (ALM), also known
as Method of Multipliers, but unlike most existing splitting methods, it only
requires access to linear (instead of quadratic) minimization oracles. We use
recent advances in the analysis of Frank-Wolfe and the alternating direction
method of multipliers algorithms to prove a sublinear convergence rate for
FW-AL over general convex compact sets and a linear convergence rate for
polytopes.

本文提出了一种名为 Frank-Wolfe Augmented Lagrangian (FW-AL) 算法的优化方法，该算法利用线性一致性约束来优化在相交凸集中的光滑函数，仅需要对单个约束的线性最小化预言机进行访问，并证明了该算法在一般凸紧集和多面体上的收敛率。