Many data distributions in the real world are hardly uniform. Instead, skewed
and long-tailed distributions of various kinds are commonly observed. This
poses an interesting problem for machine learning, where most algorithms assume
or work well with uniformly distributed data. The problem is further
exacerbated by current state-of-the-art deep learning models requiring large
volumes of training data. As such, learning from imbalanced data remains a
challenging research problem and a problem that must be solved as we move
towards more real-world applications of deep learning. In the context of class
imbalance, state-of-the-art (SOTA) accuracies on standard benchmark datasets
for classification typically fall less than 75%, even for less challenging
datasets such as CIFAR100. Nonetheless, there has been progress in this niche
area of deep learning. To this end, in this survey, we provide a taxonomy of
various methods proposed for addressing the problem of long-tail
classification, focusing on works that happened in the last few years under a
single mathematical framework. We also discuss standard performance metrics,
convergence studies, feature distribution and classifier analysis. We also
provide a quantitative comparison of the performance of different SOTA methods
and conclude the survey by discussing the remaining challenges and future
research direction.

不均匀数据的深度学习中的挑战及方法研究，重点关注长尾分类问题。

深度长尾分类进展综述

A Survey of Deep Long-Tail Classification Advancements

One of the most challenging issues in federated learning is that the data is
often not independent and identically distributed (nonIID). Clients are
expected to contribute the same type of data and drawn from one global
distribution. However, data are often collected in different ways from
different resources. Thus, the data distributions among clients might be
different from the underlying global distribution. This creates a weight
divergence issue and reduces federated learning performance. This work focuses
on improving federated learning performance for skewed data distribution across
clients. The main idea is to adjust the client distribution closer to the
global distribution using sample weights. Thus, the machine learning model
converges faster with higher accuracy. We start from the fundamental concept of
empirical risk minimization and theoretically derive a solution for adjusting
the distribution skewness using sample weights. To determine sample weights, we
implicitly exchange density information by leveraging a neural network-based
density estimation model, MADE. The clients data distribution can then be
adjusted without exposing their raw data. Our experiment results on three
real-world datasets show that the proposed method not only improves federated
learning accuracy but also significantly reduces communication costs compared
to the other experimental methods.

通过使用样本权重，将客户端的数据分布调整至全局分布，从而提高联邦学习的性能，减少通信成本。

使用样本权重进行分布倾斜数据的联邦学习

Federated Learning for distribution skewed data using sample weights

Multi-party collaborative training, such as distributed learning and
federated learning, is used to address the big data challenges. However,
traditional multi-party collaborative training algorithms were mainly designed
for balanced data mining tasks and are intended to optimize accuracy
(\emph{e.g.}, cross-entropy). The data distribution in many real-world
applications is skewed and classifiers, which are trained to improve accuracy,
perform poorly when applied to imbalanced data tasks since models could be
significantly biased toward the primary class. Therefore, the Area Under
Precision-Recall Curve (AUPRC) was introduced as an effective metric. Although
single-machine AUPRC maximization methods have been designed, multi-party
collaborative algorithm has never been studied. The change from the
single-machine to the multi-party setting poses critical challenges.
To address the above challenge, we study the serverless multi-party
collaborative AUPRC maximization problem since serverless multi-party
collaborative training can cut down the communications cost by avoiding the
server node bottleneck, and reformulate it as a conditional stochastic
optimization problem in a serverless multi-party collaborative learning setting
and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to
directly optimize the AUPRC. After that, we use the variance reduction
technique and propose ServerLess biAsed sTochastic gradiEnt with Momentum-based
variance reduction (SLATE-M) algorithm to improve the convergence rate, which
matches the best theoretical convergence result reached by the single-machine
online method. To the best of our knowledge, this is the first work to solve
the multi-party collaborative AUPRC maximization problem.

通过研究无服务器多方协同学习的 AUPRC 最大化问题，我们提出了一种新的 ServerLess 偏置随机梯度（SLATE）算法，用于直接优化 AUPRC，并使用方差缩减技术改进了收敛速度，这是目前解决多方协同 AUPRC 最大化问题的首个工作。

无服务器联合 AUPRC 优化多方协作不平衡数据挖掘

Serverless Federated AUPRC Optimization for Multi-Party Collaborative  Imbalanced Data Mining

We study the problem of computing a conjunctive query q in parallel, using p
of servers, on a large database. We consider algorithms with one round of
communication, and study the complexity of the communication. We are especially
interested in the case where the data is skewed, which is a major challenge for
scalable parallel query processing. We establish a tight connection between the
fractional edge packings of the query and the amount of communication, in two
cases. First, in the case when the only statistics on the database are the
cardinalities of the input relations, and the data is skew-free, we provide
matching upper and lower bounds (up to a poly log p factor) expressed in terms
of fractional edge packings of the query q. Second, in the case when the
relations are skewed and the heavy hitters and their frequencies are known, we
provide upper and lower bounds (up to a poly log p factor) expressed in terms
of packings of residual queries obtained by specializing the query to a heavy
hitter. All our lower bounds are expressed in the strongest form, as number of
bits needed to be communicated between processors with unlimited computational
power. Our results generalizes some prior results on uniform databases (where
each relation is a matching) [4], and other lower bounds for the MapReduce
model [1].

研究使用 p 个服务器在大型数据库上并行计算简单查询 q 的通信复杂性，特别关注数据倾斜的情况下的情况，建立了查询的分数边覆盖与通信量之间的紧密联系，并提供了查询分数边覆盖的匹配上界和下界，所有下界都以比特通信量的形式表达。