Low-Rank Adaptation (LoRA) has recently gained attention for fine-tuning
foundation models by incorporating trainable low-rank matrices, thereby
reducing the number of trainable parameters. While LoRA offers numerous
advantages, its applicability for real-time serving to a diverse and global
user base is constrained by its incapability to handle multiple task-specific
adapters efficiently. This imposes a performance bottleneck in scenarios
requiring personalized, task-specific adaptations for each incoming request. To
mitigate this constraint, we introduce Fast LoRA (FLoRA), a framework in which
each input example in a minibatch can be associated with its unique low-rank
adaptation weights, allowing for efficient batching of heterogeneous requests.
We empirically demonstrate that FLoRA retains the performance merits of LoRA,
showcasing competitive results on the MultiPL-E code generation benchmark
spanning over 8 languages and a multilingual speech recognition task across 6
languages.

通过引入名为 Fast LoRA（FLoRA）的框架，我们可以有效地对多样化和全球用户群体的实时请求进行批处理，通过将每个输入示例与其独特的低秩适应权重关联起来，实现个性化的任务特定适应，从而缓解了 Low-Rank Adaptation (LoRA) 在处理多个任务特定适配器时的性能瓶颈。我们在包括 8 种语言的 MultiPL-E 代码生成基准和 6 种语言的多语种语音识别任务上，通过实证展示了 FLoRA 保持 LoRA 性能优点的竞争结果。