Models trained with empirical risk minimization (ERM) are revealed to easily
rely on spurious correlations, resulting in poor generalization. Group
distributionally robust optimization (group DRO) can alleviate this problem by
minimizing the worst-case loss over pre-defined groups. While promising, in
practice factors like expensive annotations and privacy preclude the
availability of group labels. More crucially, when taking a closer look at the
failure modes of out-of-distribution generalization, the typical procedure of
reweighting in group DRO loses efficiency. Hinged on the limitations, in this
work, we reformulate the group DRO framework by proposing Q-Diversity.
Characterized by an interactive training mode, Q-Diversity relaxes the group
identification from annotation into direct parameterization. Furthermore, a
novel mixing strategy across groups is presented to diversify the
under-represented groups. In a series of experiments on both synthetic and
real-world text classification tasks, results demonstrate that Q-Diversity can
consistently improve worst-case accuracy under different distributional shifts,
outperforming state-of-the-art alternatives.

本研究旨在解决传统风险最小化（ERM）有偏学习的问题，通过提出基于参数化的 Q-Diversity 框架，利用交互训练模式和新型跨组混合策略实现了减小预定义组之间误差的作用，并通过实验结果验证其在文本分类任务中有效提高了最差情况下的准确性。

基于极小极大博弈的 Q-Diversity 鲁棒优化模型

Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization

Few-shot recognition learns a recognition model with very few (e.g., 1 or 5)
images per category, and current few-shot learning methods focus on improving
the average accuracy over many episodes. We argue that in real-world
applications we may often only try one episode instead of many, and hence
maximizing the worst-case accuracy is more important than maximizing the
average accuracy. We empirically show that a high average accuracy not
necessarily means a high worst-case accuracy. Since this objective is not
accessible, we propose to reduce the standard deviation and increase the
average accuracy simultaneously. In turn, we devise two strategies from the
bias-variance tradeoff perspective to implicitly reach this goal: a simple yet
effective stability regularization (SR) loss together with model ensemble to
reduce variance during fine-tuning, and an adaptability calibration mechanism
to reduce the bias. Extensive experiments on benchmark datasets demonstrate the
effectiveness of the proposed strategies, which outperforms current
state-of-the-art methods with a significant margin in terms of not only
average, but also worst-case accuracy. Our code is available at
this https URL.

该研究提出了两种策略来提高少样本识别模型的最坏情况准确率，分别是模型稳定性正则化和自适应校准机制，并在基准数据集上进行了实验，取得了当前最先进方法无法匹敌的成果。

小样本识别中的极端情况问题

Worst Case Matters for Few-Shot Recognition

Domain generalization asks for models trained over a set of training
environments to perform well in unseen test environments. Recently, a series of
algorithms such as Invariant Risk Minimization (IRM) has been proposed for
domain generalization. However, Rosenfeld et al. (2021) shows that in a simple
linear data model, even if non-convexity issues are ignored, IRM and its
extensions cannot generalize to unseen environments with less than $d_s+1$
training environments, where $d_s$ is the dimension of the spurious-feature
subspace. In this paper, we propose to achieve domain generalization with
Invariant-feature Subspace Recovery (ISR). Our first algorithm, ISR-Mean, can
identify the subspace spanned by invariant features from the first-order
moments of the class-conditional distributions, and achieve provable domain
generalization with $d_s+1$ training environments under the data model of
Rosenfeld et al. (2021). Our second algorithm, ISR-Cov, further reduces the
required number of training environments to $O(1)$ using the information of
second-order moments. Notably, unlike IRM, our algorithms bypass non-convexity
issues and enjoy global convergence guarantees. Empirically, our ISRs can
obtain superior performance compared with IRM on synthetic benchmarks. In
addition, on three real-world image and text datasets, we show that both ISRs
can be used as simple yet effective post-processing methods to improve the
worst-case accuracy of (pre-)trained models against spurious correlations and
group shifts.

本文提出了利用不变特征子空间恢复（ISR）实现域泛化的算法，ISR-Mean 和 ISR-Cov 可以从一阶和二阶矩中识别不变特征所涵盖的子空间并在训练集为 d_s+1 时实现可证明的域泛化，相比 Invariant Risk Minimization (IRM) , ISR 算法避免了非凸性问题，并具有全局收敛性保证，实验结果表明 ISR 算法在合成基准测试和图像与文本三个真实数据集上，都能作为简单但有效的后处理方法来提高训练模型对假冗余和组偏差的最坏情况准确性。