The rise in internet usage has led to the generation of massive amounts of
data, resulting in the adoption of various supervised and semi-supervised
machine learning algorithms, which can effectively utilize the colossal amount
of data to train models. However, before deploying these models in the real
world, these must be strictly evaluated on performance measures like worst-case
recall and satisfy constraints such as fairness. We find that current
state-of-the-art empirical techniques offer sub-optimal performance on these
practical, non-decomposable performance objectives. On the other hand, the
theoretical techniques necessitate training a new model from scratch for each
performance objective. To bridge the gap, we propose SelMix, a selective
mixup-based inexpensive fine-tuning technique for pre-trained models, to
optimize for the desired objective. The core idea of our framework is to
determine a sampling distribution to perform a mixup of features between
samples from particular classes such that it optimizes the given objective. We
comprehensively evaluate our technique against the existing empirical and
theoretically principled methods on standard benchmark datasets for imbalanced
classification. We find that proposed SelMix fine-tuning significantly improves
the performance for various practical non-decomposable objectives across
benchmarks.

通过互联网使用量的增加产生了大量的数据，从而导致了各种监督和半监督机器学习算法的采用，这些算法可以有效利用庞大的数据量来训练模型。然而，在将这些模型部署到实际环境之前，必须严格评估性能指标（如最坏情况召回率）并满足公平性等约束。我们发现目前最先进的经验技术在这些实际的不可分解性能目标上提供了次优的性能。另一方面，理论技术需要针对每个性能目标从头开始训练新模型。为了弥合这一差距，我们提出了 SelMix 技术，这是一种选择性混合的低成本微调技术，用于已预训练模型以优化所需的目标。我们的核心思想是确定一个采样分布，以在特定类别之间的样本之间进行特征混合，从而优化给定的目标。我们在标准基准数据集上对我们的技术进行了全面评估，与现有的经验和理论基础方法进行了比较，用于不平衡分类。我们发现所提出的 SelMix 微调显著提高了在各种实际的不可分解目标上的性能。