Many optimization problems are inherently multi-objective. To address them,
we formalize Jacobian descent (JD), a direct generalization of gradient descent
for vector-valued functions. Each step of this algorithm relies on a Jacobian
matrix consisting of one gradient per objective. The aggregator, responsible
for reducing this matrix into an update vector, characterizes JD. While the
multi-task learning literature already contains a variety of aggregators, they
often lack some natural properties. In particular, the update should not
conflict with any objective and should scale proportionally to the norm of each
gradient. We propose a new aggregator specifically designed to satisfy this.
Emphasizing conflict between objectives, we then highlight direct applications
for our methods. Most notably, we introduce instance-wise risk minimization
(IWRM), a learning paradigm in which the loss of each training example is
considered a separate objective. On simple image classification tasks, IWRM
exhibits promising results compared to the direct minimization of the average
loss. The performance of our aggregator in those experiments also corroborates
our theoretical findings. Lastly, as speed is the main limitation of JD, we
provide a path towards a more efficient implementation.

我们提出了一种新的多目标优化算法，基于 Jacobian 下降方法，并设计了一个特定的汇集器来确保更新向量不与任何目标冲突，并且与每个梯度的范数成比例；我们在简单的图像分类任务中引入了实例风险最小化学习方法，实验展示了该方法相比于平均损失的直接最小化的有希望的结果，并且验证了我们理论上的发现，最后，为了提高 Jacobian 下降的速度，我们提供了更高效实现的路径。

雅可比下降多目标优化

Jacobian Descent for Multi-Objective Optimization

Federated leaning (FL) has been frequently used in various field of studies
and businesses. Traditional centralized FL systems suffer from serious issues.
To address these concerns, decentralized federated learning (DFL) systems have
been introduced in recent years in which with the help of blockchains, try to
achieve more integrity and efficiency. On the other hand, privacy-preserving is
an uncovered part of these systems. To address this, and also scaling the
blockchain-based computations, we propose a zero knowledge proof (ZKP) based
aggregator (zkDFL) that allows clients to share their large-scale model
parameters with a trusted centralized server without revealing their individual
data to other clients. We utilize blockchain technology to manage the
aggregation algorithm via smart contracts. The server performs a ZKP algorithm
to prove to the clients that the aggregation is done according to the accepted
algorithm. The server can also prove that all inputs of clients have been used.
We evaluate our measure through a public dataset about wearable internet of
things. As demonstrated by numerical evaluations, zkDFL introduces
verifiability of correctness of aggregation process and enhances the privacy
protection and scalability of DFL systems, while the gas cost has declined
significantly.

基于零知识证明的聚合器 (zkDFL) 使用区块链技术管理聚合算法，以验证进程正确性、增强隐私保护和可扩展性，同时显著降低了燃气成本。