Optimizing the performance of many objectives (instantiated by tasks or
clients) jointly with a few Pareto stationary solutions (models) is critical in
machine learning. However, previous multi-objective optimization methods often
focus on a few number of objectives and cannot scale to many objectives that
outnumber the solutions, leading to either subpar performance or ignored
objectives. We introduce Many-objective multi-solution Transport (MosT), a
framework that finds multiple diverse solutions in the Pareto front of many
objectives. Our insight is to seek multiple solutions, each performing as a
domain expert and focusing on a specific subset of objectives while
collectively covering all of them. MosT formulates the problem as a bi-level
optimization of weighted objectives for each solution, where the weights are
defined by an optimal transport between the objectives and solutions. Our
algorithm ensures convergence to Pareto stationary solutions for complementary
subsets of objectives. On a range of applications in federated learning,
multi-task learning, and mixture-of-prompt learning for LLMs, MosT distinctly
outperforms strong baselines, delivering high-quality, diverse solutions that
profile the entire Pareto frontier, thus ensuring balanced trade-offs across
many objectives.

通过寻求多个多目标优化问题的互补子集解决方案的权衡，我们引入了一种称为 MosT 的多目标多解传输框架，其通过在每个解决方案中对加权目标进行双层优化，并使用目标和解决方案之间的最优传输来定义权重，确保收敛到 Pareto 稳态解决方案。在联邦学习、多任务学习和混合提示学习等应用中，MosT 都显著优于强基线方法，提供高质量的多样化解决方案，刻画了整个 Pareto 前沿，从而实现了在多个目标之间的平衡权衡。

多目标多解输送

Many-Objective Multi-Solution Transport

Fairness has emerged as a critical problem in federated learning (FL). In
this work, we identify a cause of unfairness in FL -- conflicting gradients
with large differences in the magnitudes. To address this issue, we propose the
federated fair averaging (FedFV) algorithm to mitigate potential conflicts
among clients before averaging their gradients. We first use the cosine
similarity to detect gradient conflicts, and then iteratively eliminate such
conflicts by modifying both the direction and the magnitude of the gradients.
We further show the theoretical foundation of FedFV to mitigate the issue
conflicting gradients and converge to Pareto stationary solutions. Extensive
experiments on a suite of federated datasets confirm that FedFV compares
favorably against state-of-the-art methods in terms of fairness, accuracy and
efficiency. The source code is available at this https URL

本文探讨联邦学习中不公平性的问题，提出了解决方案 ——FedFV 算法，并且基于余弦相似性检测梯度冲突，在梯度平均之前消除这些冲突。实验证明，FedFV 在公平性、准确性和效率方面比现有方法表现更好，并能收敛于 Pareto 稳态解。