Learned reweighting (LRW) approaches to supervised learning use an
optimization criterion to assign weights for training instances, in order to
maximize performance on a representative validation dataset. We pose and
formalize the problem of optimized selection of the validation set used in LRW
training, to improve classifier generalization. In particular, we show that
using hard-to-classify instances in the validation set has both a theoretical
connection to, and strong empirical evidence of generalization. We provide an
efficient algorithm for training this meta-optimized model, as well as a simple
train-twice heuristic for careful comparative study. We demonstrate that LRW
with easy validation data performs consistently worse than LRW with hard
validation data, establishing the validity of our meta-optimization problem.
Our proposed algorithm outperforms a wide range of baselines on a range of
datasets and domain shift challenges (Imagenet-1K, CIFAR-100, Clothing-1M,
CAMELYON, WILDS, etc.), with ~1% gains using VIT-B on Imagenet. We also show
that using naturally hard examples for validation (Imagenet-R / Imagenet-A) in
LRW training for Imagenet improves performance on both clean and naturally hard
test instances by 1-2%. Secondary analyses show that using hard validation data
in an LRW framework improves margins on test data, hinting at the mechanism
underlying our empirical gains. We believe this work opens up new research
directions for the meta-optimization of meta-learning in a supervised learning
context.

学习重加权（LRW）方法用优化准则分配训练实例权重，以在代表性验证数据集上最大化性能。我们提出并形式化了在 LRW 训练中优化选择验证集的问题，以提高分类器的泛化能力。使用难以分类的实例作为验证集，在理论上与实证的有效性存在强烈的关联。我们提供了一种有效的算法来训练这种元优化模型，并提出了一种简单的两次训练启发式方法进行比较研究。我们证明相对于易用验证数据的 LRW，具有难度验证数据的 LRW 表现一致更差，从而验证了我们元优化问题的有效性。我们的算法在一系列数据集和域偏移挑战（Imagenet-1K，CIFAR-100，Clothing-1M，CAMELYON，WILDS 等）上优于广泛的基线，通过在 Imagenet 上使用 VIT-B 获得了约 1％的增益。我们还展示了在 LRW 的训练中使用自然难的示例（Imagenet-R/Imagenet-A）能够在干净和自然难的测试实例上分别提高 1-2％的性能。次要分析显示，在 LRW 框架中使用难度验证数据可以提高测试数据的边界，暗示了我们实证增益背后的机制。我们认为这项工作为监督学习环境中元学习的元优化开辟了新的研究方向。

通过在难样本上进行元学习以提高泛化能力

Improving Generalization via Meta-Learning on Hard Samples

An increasing number of applications require to recognize the class of an
incoming time series as quickly as possible without unduly compromising the
accuracy of the prediction. In this paper, we put forward a new optimization
criterion which takes into account both the cost of misclassification and the
cost of delaying the decision. Based on this optimization criterion, we derived
a family of non-myopic algorithms which try to anticipate the expected future
gain in information in balance with the cost of waiting. In one class of
algorithms, unsupervised-based, the expectations use the clustering of time
series, while in a second class, supervised-based, time series are grouped
according to the confidence level of the classifier used to label them.
Extensive experiments carried out on real data sets using a large range of
delay cost functions show that the presented algorithms are able to
satisfactorily solving the earliness vs. accuracy trade-off, with the
supervised-based approaches faring better than the unsupervised-based ones. In
addition, all these methods perform better in a wide variety of conditions than
a state of the art method based on a myopic strategy which is recognized as
very competitive.

本文提出了一种采用新的优化准则的分类算法，该准则考虑了分类错误的成本和等待决策的成本，并尝试通过使用聚类或分类器来减少误判和延迟的影响，实验结果表明，这种方法优于传统的基于经验策略的方法。

时间序列的早期分类。基于成本的优化标准和算法

Early Classification of Time Series. Cost-based Optimization Criterion  and Algorithms

Item recommendation is the task of predicting a personalized ranking on a set
of items (e.g. websites, movies, products). In this paper, we investigate the
most common scenario with implicit feedback (e.g. clicks, purchases). There are
many methods for item recommendation from implicit feedback like matrix
factorization (MF) or adaptive knearest-neighbor (kNN). Even though these
methods are designed for the item prediction task of personalized ranking, none
of them is directly optimized for ranking. In this paper we present a generic
optimization criterion BPR-Opt for personalized ranking that is the maximum
posterior estimator derived from a Bayesian analysis of the problem. We also
provide a generic learning algorithm for optimizing models with respect to
BPR-Opt. The learning method is based on stochastic gradient descent with
bootstrap sampling. We show how to apply our method to two state-of-the-art
recommender models: matrix factorization and adaptive kNN. Our experiments
indicate that for the task of personalized ranking our optimization method
outperforms the standard learning techniques for MF and kNN. The results show
the importance of optimizing models for the right criterion.

本文研究了使用隐式反馈（点击、购买等）的个性化推荐任务，提出了一种基于贝叶斯分析的最大后验估计通用优化准则（BPR-Opt）和基于随机梯度下降与自助抽样的通用学习算法。实验证明，该方法优于矩阵分解和自适应 k 最近邻两种先前的标准学习技术。