While directly fine-tuning (FT) large-scale, pretrained models on task-specific data is well-known to induce strong in-distribution task performance, recent works have demonstrated that different adaptation protocols, such as linear probing (LP) prior to FT, can improve out-of-distribution generalization. However, the design space of such adaptation protocols remains under-explored and the evaluation of such protocols has primarily focused on distribution shifts. Therefore, in this work, we evaluate common adaptation protocols across distributions shifts and machine learning safety metrics (e.g., anomaly detection, calibration, robustness to corruptions). We find that protocols induce disparate trade-offs that were not apparent from prior evaluation. Further, we demonstrate that appropriate pairing of data augmentation and protocol can substantially mitigate this trade-off. Finally, we hypothesize and empirically see that using hardness-promoting augmentations during LP and then FT with augmentations may be particularly effective for trade-off mitigation.

本研究通过评估常见的适应协议跨分布改变和机器学习安全度量，并探讨如何应用数据扩增来减轻不同适应协议的权衡。最终，假设并验证了在线性探测时使用硬度增强扩展，然后使用扩展调整的硬度增强扩展可能特别有效来减轻权衡。

探索适应性协议的设计，以实现更好的泛化和机器学习安全