Adapters provide an efficient and lightweight mechanism for adapting trained
transformer models to a variety of different tasks. However, they have often
been found to be outperformed by other adaptation mechanisms, including
low-rank adaptation. In this paper, we provide an in-depth study of adapters,
their internal structure, as well as various implementation choices. We uncover
pitfalls for using adapters and suggest a concrete, improved adapter
architecture, called Adapter+, that not only outperforms previous adapter
implementations but surpasses a number of other, more complex adaptation
mechanisms in several challenging settings. Despite this, our suggested adapter
is highly robust and, unlike previous work, requires little to no manual
intervention when addressing a novel scenario. Adapter+ reaches
state-of-the-art average accuracy on the VTAB benchmark, even without a
per-task hyperparameter optimization.

适配器提供了一个高效轻量级的机制，可以将训练好的变换器模型适应不同的任务。本文对适配器进行了深入研究，包括其内部结构和各种实现选择，并提出了一种改进的适配器架构，称为 Adapter+，它不仅优于之前的适配器实现，还在一些具有挑战性的场景中超越了其他更复杂的适应机制。与之前的工作不同，我们的 Adapter + 在应对新的情景时几乎不需要人工干预，具有极高的鲁棒性。即使没有针对每个任务的超参数优化，Adapter + 也能在 VTAB 基准测试中达到最先进的平均准确率。