The performance of machine learning models can significantly degrade under distribution shifts of the data. We propose a new method for classification which can improve robustness to distribution shifts, by combining expert knowledge about the ``high-level" structure of the data with standard classifiers. Specifically, we introduce two-stage classifiers called \textit{memory classifiers}. First, these identify prototypical data points -- \textit{memories} -- to cluster the training data. This step is based on features designed with expert guidance; for instance, for image data they can be extracted using digital image processing algorithms. Then, within each cluster, we learn local classifiers based on finer discriminating features, via standard models like deep neural networks. We establish generalization bounds for memory classifiers. We illustrate in experiments that they can improve generalization and robustness to distribution shifts on image datasets. We show improvements which push beyond standard data augmentation techniques.

本研究提出了一种新的分类方法，称为Memory Classifiers，通过结合高级数据结构和专家知识，识别出典型数据点，再在每个类别中学习细分特征的本地分类器，以提高机器学习模型的鲁棒性和泛化能力，实验证明，该方法可超越标准数据增强技术。

记忆分类器：机器学习中的鲁棒性双阶段分类