Many optimization problems are inherently multi-objective. To address them,
we formalize Jacobian descent (JD), a direct generalization of gradient descent
for vector-valued functions. Each step of this algorithm relies on a Jacobian
matrix consisting of one gradient per objective. The aggregator, responsible
for reducing this matrix into an update vector, characterizes JD. While the
multi-task learning literature already contains a variety of aggregators, they
often lack some natural properties. In particular, the update should not
conflict with any objective and should scale proportionally to the norm of each
gradient. We propose a new aggregator specifically designed to satisfy this.
Emphasizing conflict between objectives, we then highlight direct applications
for our methods. Most notably, we introduce instance-wise risk minimization
(IWRM), a learning paradigm in which the loss of each training example is
considered a separate objective. On simple image classification tasks, IWRM
exhibits promising results compared to the direct minimization of the average
loss. The performance of our aggregator in those experiments also corroborates
our theoretical findings. Lastly, as speed is the main limitation of JD, we
provide a path towards a more efficient implementation.

我们提出了一种新的多目标优化算法，基于 Jacobian 下降方法，并设计了一个特定的汇集器来确保更新向量不与任何目标冲突，并且与每个梯度的范数成比例；我们在简单的图像分类任务中引入了实例风险最小化学习方法，实验展示了该方法相比于平均损失的直接最小化的有希望的结果，并且验证了我们理论上的发现，最后，为了提高 Jacobian 下降的速度，我们提供了更高效实现的路径。