We address the problem of learning on sets of features, motivated by the need
of performing pooling operations in long biological sequences of varying sizes,
with long-range dependencies, and possibly few labeled data. To address this
challenging task, we introduce a parametrized representation of fixed size,
which embeds and then aggregates elements from a given input set according to
the optimal transport plan between the set and a trainable reference. Our
approach scales to large datasets and allows end-to-end training of the
reference, while also providing a simple unsupervised learning mechanism with
small computational cost. Our aggregation technique admits two useful
interpretations: it may be seen as a mechanism related to attention layers in
neural networks, or it may be seen as a scalable surrogate of a classical
optimal transport-based kernel. We experimentally demonstrate the effectiveness
of our approach on biological sequences, achieving state-of-the-art results for
protein fold recognition and detection of chromatin profiles tasks, and, as a
proof of concept, we show promising results for processing natural language
sequences. We provide an open-source implementation of our embedding that can
be used alone or as a module in larger learning models at
this https URL

本文提出了一种基于优化传输计划思想的特征集聚合技术，可用于长生物序列的池化操作，具有快速、简单、可扩展等优点，并在蛋白质折叠识别、染色质图谱检测等生物学任务上取得了最先进的效果。

一个可训练的最优输运嵌入及其与注意力的关系

A Trainable Optimal Transport Embedding for Feature Aggregation and its  Relationship to Attention

Consider the problem of optimally matching two measures on the circle, or
equivalently two periodic measures on the real line, and suppose the cost of
matching two points satisfies the Monge condition. We introduce a notion of
locally optimal transport plan, motivated by the weak KAM (Aubry-Mather)
theory, and show that all locally optimal transport plans are conjugate to
shifts and that the cost of a locally optimal transport plan is a convex
function of a shift parameter. This theory is applied to a transportation
problem arising in image processing: for two sets of point masses on the
circle, both of which have the same total mass, find an optimal transport plan
with respect to a given cost function satisfying the Monge condition. In the
circular case the sorting strategy fails to provide a unique candidate solution
and a naive approach requires a quadratic number of operations. For the case of
$N$ real-valued point masses we present an O(N |log epsilon|) algorithm that
approximates the optimal cost within epsilon; when all masses are integer
multiples of 1/M, the algorithm gives an exact solution in O(N log M)
operations.

研究采用局部优化运输计划的理论进行解决图像处理的问题。得到所有局部最优输运计划都与移动呈共轭关系的结论，并验证了局部最优输运计划的费用是以移动参数的凸函数。提出了一种算法可以近似优化费用，当所有质量是 1/M 的整数倍时，该算法可以在 O (NlogM) 操作中实现准确解决方案。