This paper introduces the award-winning deep learning (DL) library called
LibAUC for implementing state-of-the-art algorithms towards optimizing a family
of risk functions named X-risks. X-risks refer to a family of compositional
functions in which the loss function of each data point is defined in a way
that contrasts the data point with a large number of others. They have broad
applications in AI for solving classical and emerging problems, including but
not limited to classification for imbalanced data (CID), learning to rank
(LTR), and contrastive learning of representations (CLR). The motivation of
developing LibAUC is to address the convergence issues of existing libraries
for solving these problems. In particular, existing libraries may not converge
or require very large mini-batch sizes in order to attain good performance for
these problems, due to the usage of the standard mini-batch technique in the
empirical risk minimization (ERM) framework. Our library is for deep X-risk
optimization (DXO) that has achieved great success in solving a variety of
tasks for CID, LTR and CLR. The contributions of this paper include: (1) It
introduces a new mini-batch based pipeline for implementing DXO algorithms,
which differs from existing DL pipeline in the design of controlled data
samplers and dynamic mini-batch losses; (2) It provides extensive benchmarking
experiments for ablation studies and comparison with existing libraries. The
LibAUC library features scalable performance for millions of items to be
contrasted, faster and better convergence than existing libraries for
optimizing X-risks, seamless PyTorch deployment and versatile APIs for various
loss optimization. Our library is available to the open source community at
this https URL, to facilitate further academic
research and industrial applications.

本文介绍了一个用于优化风险函数家族 X-risks 的深度学习库 LibAUC，它采用一种新的 mini-batch 管道实现 DXO 算法，实现了 CID、LTR 和 CLR 等多种任务的良好性能和收敛性。

LibAUC: 一个用于 X - 风险优化的深度学习库

LibAUC: A Deep Learning Library for X-Risk Optimization

The article examines in some detail the convergence rate and
mean-square-error performance of momentum stochastic gradient methods in the
constant step-size and slow adaptation regime. The results establish that
momentum methods are equivalent to the standard stochastic gradient method with
a re-scaled (larger) step-size value. The size of the re-scaling is determined
by the value of the momentum parameter. The equivalence result is established
for all time instants and not only in steady-state. The analysis is carried out
for general strongly convex and smooth risk functions, and is not limited to
quadratic risks. One notable conclusion is that the well-known bene ts of
momentum constructions for deterministic optimization problems do not
necessarily carry over to the adaptive online setting when small constant
step-sizes are used to enable continuous adaptation and learn- ing in the
presence of persistent gradient noise. From simulations, the equivalence
between momentum and standard stochastic gradient methods is also observed for
non-differentiable and non-convex problems.

研究了动量随机梯度法在常数步长和慢适应区域的收敛速度和均方误差性能，结果表明动量方法相当于具有重新缩放（更大）步长值的标准随机梯度法，重新缩放的大小由动量参数的值确定。分析适用于一般的强凸和光滑风险函数，而不限于二次风险，发现动量构建在确定性优化问题中的优点并不一定适用于小常数步长的自适应在线设置中，以使根据持续的梯度噪声进行连续适应和学习。模拟结果表明，动量和标准随机梯度方法在非可微和非凸问题中也是等价的。