BriefGPT.xyz
Mar, 2024
Arcee的合并工具包:一个用于合并大型语言模型的工具包
Arcee's MergeKit: A Toolkit for Merging Large Language Models
HTML
PDF
Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin...
TL;DR
采用开源语言模型、迁移学习和模型合并技术,通过创建多任务模型提升性能和应用领域的研究。为了支持这一领域的发展,推出了名为MergeKit的开源库,该库提供了一个可扩展的框架,便于在任何硬件上高效合并模型。
Abstract
The rapid expansion of the
open-source language model
landscape presents an opportunity to merge the competencies of these model checkpoints by combining their parameters. Advances in
transfer learning
, the proce
→