Processing sets or other unordered, potentially variable-sized inputs in
neural networks is usually handled by aggregating a number of input tensors
into a single representation. While a number of aggregation methods already
exist from simple sum pooling to multi-head attention, they are limited in
their representational power both from theoretical and empirical perspectives.
On the search of a principally more powerful aggregation strategy, we propose
an optimization-based method called Equilibrium Aggregation. We show that many
existing aggregation methods can be recovered as special cases of Equilibrium
Aggregation and that it is provably more efficient in some important cases.
Equilibrium Aggregation can be used as a drop-in replacement in many existing
architectures and applications. We validate its efficiency on three different
tasks: median estimation, class counting, and molecular property prediction. In
all experiments, Equilibrium Aggregation achieves higher performance than the
other aggregation techniques we test.

提出一种名为 Equilibrium Aggregation 的优化方法，可作为许多现有架构和应用程序中的替代方法，有效地处理神经网络中的输入聚合，达到了比其他聚合技术更高的性能。