Modular and composable transfer learning is an emerging direction in the
field of Parameter Efficient Fine-Tuning, as it enables neural networks to
better organize various aspects of knowledge, leading to improved cross-task
generalization. In this paper, we introduce a novel approach Customized
Polytropon C-Poly that combines task-common skills and task-specific skills,
while the skill parameters being highly parameterized using low-rank
techniques. Each task is associated with a customizable number of exclusive
specialized skills and also benefits from skills shared with peer tasks. A
skill assignment matrix is jointly learned. To evaluate our approach, we
conducted extensive experiments on the Super-NaturalInstructions and the
SuperGLUE benchmarks. Our findings demonstrate that C-Poly outperforms
fully-shared, task-specific, and skill-indistinguishable baselines,
significantly enhancing the sample efficiency in multi-task learning scenarios.

模块化和可组合的迁移学习是参数高效微调领域的新兴方向，它使神经网络能更好地组织各种知识方面，提高跨任务泛化能力；本文介绍了一种新的方法 Customized Polytropon C-Poly，它结合了任务共同技能和任务特定技能，并使用低秩技术对技能参数进行高度参数化；每个任务关联着可定制数量的专属专业技能，并从与同伴任务共享的技能中受益；共同学习了技能分配矩阵；通过在 Super-NaturalInstructions 和 SuperGLUE 基准上进行广泛实验评估，证明了 C-Poly 优于完全共享，任务特定和技能不可区分的基线方法，显著提高了多任务学习场景下的样本效率。

多任务学习的参数高效模块可定制组合

Customizable Combination of Parameter-Efficient Modules for Multi-Task  Learning

We introduce Adapters, an open-source library that unifies
parameter-efficient and modular transfer learning in large language models. By
integrating 10 diverse adapter methods into a unified interface, Adapters
offers ease of use and flexible configuration. Our library allows researchers
and practitioners to leverage adapter modularity through composition blocks,
enabling the design of complex adapter setups. We demonstrate the library's
efficacy by evaluating its performance against full fine-tuning on various NLP
tasks. Adapters provides a powerful tool for addressing the challenges of
conventional fine-tuning paradigms and promoting more efficient and modular
transfer learning. The library is available via this https URL

Adapters 是一个开源库，将参数高效和模块化的迁移学习统一起来，通过整合 10 种不同的适配器方法，提供简便的使用和灵活的配置；该库通过组合模块允许研究人员和从业者利用适配器的模块化设计复杂的适配器设置；通过在各种自然语言处理任务上评估其性能，我们展示了该库的有效性；Adapters 为解决传统微调范式的挑战和促进更高效和模块化的迁移学习提供了一个强大的工具；该库可通过此链接获取。