Energy-Based Models (EBMs) are known in the Machine Learning community for
the decades. Since the seminal works devoted to EBMs dating back to the
noughties there have been appearing a lot of efficient methods which solve the
generative modelling problem by means of energy potentials (unnormalized
likelihood functions). In contrast, the realm of Optimal Transport (OT) and, in
particular, neural OT solvers is much less explored and limited by few recent
works (excluding WGAN based approaches which utilize OT as a loss function and
do not model OT maps themselves). In our work, we bridge the gap between EBMs
and Entropy-regularized OT. We present the novel methodology which allows
utilizing the recent developments and technical improvements of the former in
order to enrich the latter. We validate the applicability of our method on toy
2D scenarios as well as standard unpaired image-to-image translation problems.
For the sake of simplicity, we choose simple short- and long- run EBMs as a
backbone of our Energy-guided Entropic OT method, leaving the application of
more sophisticated EBMs for future research.

该研究桥接了 EBMs 和熵正则化 OT 之间的差距，并提出了新的方法，可在玩具 2D 场景和标准无配对图像到图像转换问题上应用。

基于能量引导的熵正则神经最优输运

Energy-guided Entropic Neural Optimal Transport

Regularized optimal transport (OT) is now increasingly used as a loss or as a
matching layer in neural networks. Entropy-regularized OT can be computed using
the Sinkhorn algorithm but it leads to fully-dense transportation plans,
meaning that all sources are (fractionally) matched with all targets. To
address this issue, several works have investigated quadratic regularization
instead. This regularization preserves sparsity and leads to unconstrained and
smooth (semi) dual objectives, that can be solved with off-the-shelf gradient
methods. Unfortunately, quadratic regularization does not give direct control
over the cardinality (number of nonzeros) of the transportation plan. We
propose in this paper a new approach for OT with explicit cardinality
constraints on the transportation plan. Our work is motivated by an application
to sparse mixture of experts, where OT can be used to match input tokens such
as image patches with expert models such as neural networks. Cardinality
constraints ensure that at most $k$ tokens are matched with an expert, which is
crucial for computational performance reasons. Despite the nonconvexity of
cardinality constraints, we show that the corresponding (semi) dual problems
are tractable and can be solved with first-order gradient methods. Our method
can be thought as a middle ground between unregularized OT (recovered in the
limit case $k=1$) and quadratically-regularized OT (recovered when $k$ is large
enough). The smoothness of the objectives increases as $k$ increases, giving
rise to a trade-off between convergence speed and sparsity of the optimal plan.

本文提出了一种新的最优输运问题方法，具有明确的基数约束，既能保证传送方案的稀疏性，又能够限制匹配的最大数量，使其在稀疏专家混合等应用中能够更好地解决计算性能问题。

稀疏约束的最优输运

Sparsity-Constrained Optimal Transport

Optimal transport (OT) formalizes the problem of finding an optimal coupling
between probability measures given a cost matrix. The inverse problem of
inferring the cost given a coupling is Inverse Optimal Transport (IOT). IOT is
less well understood than OT. We formalize and systematically analyze the
properties of IOT using tools from the study of entropy-regularized OT.
Theoretical contributions include characterization of the manifold of
cross-ratio equivalent costs, the implications of model priors, and derivation
of an MCMC sampler. Empirical contributions include visualizations of
cross-ratio equivalent effect on basic examples and simulations validating
theoretical results.

本文通过熵正则化的 Optimal transport（OT）工具，对 Inverse Optimal Transport（IOT）进行正式化和系统分析，包括代价等价成本的流形特性、模型先验的影响以及基于模拟的结果验证等方面。