The loss function plays an important role in optimizing the performance of a
learning system. A crucial aspect of the loss function is the assignment of
sample weights within a mini-batch during loss computation. In the context of
continual learning (CL), most existing strategies uniformly treat samples when
calculating the loss value, thereby assigning equal weights to each sample.
While this approach can be effective in certain standard benchmarks, its
optimal effectiveness, particularly in more complex scenarios, remains
underexplored. This is particularly pertinent in training "in the wild," such
as with self-training, where labeling is automated using a reference model.
This paper introduces the Online Meta-learning for Sample Importance (OMSI)
strategy that approximates sample weights for a mini-batch in an online CL
stream using an inner- and meta-update mechanism. This is done by first
estimating sample weight parameters for each sample in the mini-batch, then,
updating the model with the adapted sample weights. We evaluate OMSI in two
distinct experimental settings. First, we show that OMSI enhances both learning
and retained accuracy in a controlled noisy-labeled data stream. Then, we test
the strategy in three standard benchmarks and compare it with other popular
replay-based strategies. This research aims to foster the ongoing exploration
in the area of self-adaptive CL.

本研究介绍了一种在线连续学习中使用样本重要性进行样本权重估计的策略，通过适应样本权重来提高学习性能和保留准确性。研究通过在噪声标注的数据流和标准基准测试中进行评估，旨在促进自适应连续学习领域的深入探索。

使用元更新进行在线连续学习的样本权重估计

Sample Weight Estimation Using Meta-Updates for Online Continual  Learning

One of the most challenging issues in federated learning is that the data is
often not independent and identically distributed (nonIID). Clients are
expected to contribute the same type of data and drawn from one global
distribution. However, data are often collected in different ways from
different resources. Thus, the data distributions among clients might be
different from the underlying global distribution. This creates a weight
divergence issue and reduces federated learning performance. This work focuses
on improving federated learning performance for skewed data distribution across
clients. The main idea is to adjust the client distribution closer to the
global distribution using sample weights. Thus, the machine learning model
converges faster with higher accuracy. We start from the fundamental concept of
empirical risk minimization and theoretically derive a solution for adjusting
the distribution skewness using sample weights. To determine sample weights, we
implicitly exchange density information by leveraging a neural network-based
density estimation model, MADE. The clients data distribution can then be
adjusted without exposing their raw data. Our experiment results on three
real-world datasets show that the proposed method not only improves federated
learning accuracy but also significantly reduces communication costs compared
to the other experimental methods.

通过使用样本权重，将客户端的数据分布调整至全局分布，从而提高联邦学习的性能，减少通信成本。

使用样本权重进行分布倾斜数据的联邦学习

Federated Learning for distribution skewed data using sample weights

Most fair machine learning methods either highly rely on the sensitive
information of the training samples or require a large modification on the
target models, which hinders their practical application. To address this
issue, we propose a two-stage training algorithm named FAIRIF. It minimizes the
loss over the reweighted data set (second stage) where the sample weights are
computed to balance the model performance across different demographic groups
(first stage). FAIRIF can be applied on a wide range of models trained by
stochastic gradient descent without changing the model, while only requiring
group annotations on a small validation set to compute sample weights.
Theoretically, we show that, in the classification setting, three notions of
disparity among different groups can be mitigated by training with the weights.
Experiments on synthetic data sets demonstrate that FAIRIF yields models with
better fairness-utility trade-offs against various types of bias; and on
real-world data sets, we show the effectiveness and scalability of FAIRIF.
Moreover, as evidenced by the experiments with pretrained models, FAIRIF is
able to alleviate the unfairness issue of pretrained models without hurting
their performance.

我们提出了一个名为 FAIRIF 的两阶段训练算法，它可以应用于一系列使用随机梯度下降训练的模型，并通过在小型验证集上计算样本权重来实现跨不同人口群体的模型性能平衡，从而缓解先前机器学习算法中存在的不公平问题。在合成数据集和实际数据集上的实验表明，FAIRIF 可以在牺牲很少的模型实用性的情况下实现更好的公平性 - 实用性权衡。

通过平衡影响训练公平的深度神经网络

Training Fair Deep Neural Networks by Balancing Influence

We propose a simple method by which to choose sample weights for problems
with highly imbalanced or skewed traits. Rather than naively discretizing
regression labels to find binned weights, we take a more principled approach --
we derive sample weights from the transfer function between an estimated source
and specified target distributions. Our method outperforms both unweighted and
discretely-weighted models on both regression and classification tasks. We also
open-source our implementation of this method
(this https URL) to the scientific
community.

通过从预测源分布和指定目标分布之间的转移函数中得出样本权重，我们提出了一种简单的方法来选择高度不平衡或偏斜特征问题的样本权重。该方法在回归和分类任务上优于无权重和离散权重模型，并把该算法实现开源给科学界。