BriefGPT.xyz
Apr, 2024
医学基础模型的低秩知识分解
Low-Rank Knowledge Decomposition for Medical Foundation Models
HTML
PDF
Yuhang Zhou, Haolin Li, Siyuan Du, Jiangchao Yao, Ya Zhang...
TL;DR
本文通过知识分解的方法,设计了一种名为Low-Rank Knowledge Decomposition (LoRKD)的新型框架,将医学基础模型拆分为多个轻量级专家模型,以提高特定医学任务的性能和专业化,并在资源消耗上达到平衡。实验结果表明,拆分后的模型在性能和可迁移性方面表现良好,甚至超过原始的基础模型。
Abstract
The popularity of
large-scale pre-training
has promoted the development of
medical foundation models
. However, some studies have shown that although foundation models exhibit strong general feature extraction cap
→