Robust fine-tuning aims to achieve competitive in-distribution (ID)
performance while maintaining the out-of-distribution (OOD) robustness of a
pre-trained model when transferring it to a downstream task. Recently,
projected gradient descent has been successfully used in robust fine-tuning by
constraining the deviation from the initialization of the fine-tuned model
explicitly through projection. However, algorithmically, two limitations
prevent this method from being adopted more widely, scalability and efficiency.
In this paper, we propose a new projection-based fine-tuning algorithm, Fast
Trainable Projection (FTP) for computationally efficient learning of per-layer
projection constraints, resulting in an average $35\%$ speedup on our
benchmarks compared to prior works. FTP can be combined with existing
optimizers such as AdamW, and be used in a plug-and-play fashion. Finally, we
show that FTP is a special instance of hyper-optimizers that tune the
hyper-parameters of optimizers in a learnable manner through nested
differentiation. Empirically, we show superior robustness on OOD datasets,
including domain shifts and natural corruptions, across four different vision
tasks with five different pre-trained models. Additionally, we demonstrate that
FTP is broadly applicable and beneficial to other learning scenarios such as
low-label and continual learning settings thanks to its easy adaptability. The
code will be available at this https URL

通过投影的方法，我们提出了一种快速可训练的投影算法（FTP）来实现鲁棒精调，该算法能够快速学习每层投影约束，相较于之前的工作，我们的基准测试中速度提升了 35％。我们通过对四个不同视觉任务和五个不同的预训练模型进行的经验性研究表明，FTP 在 OOD 数据集，包括领域转移和自然失真方面表现出优越鲁棒性。此外，我们展示了 FTP 在低标签和持续学习设置等其他学习场景中的广泛适用性和益处。