Beyond scaling base models with more data or parameters, fine-tuned adapters
provide an alternative way to generate high fidelity, custom images at reduced
costs. As such, adapters have been widely adopted by open-source communities,
accumulating a database of over 100K adapters-most of which are highly
customized with insufficient descriptions. This paper explores the problem of
matching the prompt to a set of relevant adapters, built on recent work that
highlight the performance gains of composing adapters. We introduce Stylus,
which efficiently selects and automatically composes task-specific adapters
based on a prompt's keywords. Stylus outlines a three-stage approach that first
summarizes adapters with improved descriptions and embeddings, retrieves
relevant adapters, and then further assembles adapters based on prompts'
keywords by checking how well they fit the prompt. To evaluate Stylus, we
developed StylusDocs, a curated dataset featuring 75K adapters with
pre-computed adapter embeddings. In our evaluation on popular Stable Diffusion
checkpoints, Stylus achieves greater CLIP-FID Pareto efficiency and is twice as
preferred, with humans and multimodal models as evaluators, over the base
model. See stylus-diffusion.github.io for more.

通过关键词选择和自动组合任务特定的适配器，Stylus 提供了一种高效的方法来匹配提示与相关适配器，从而在生成定制图像方面节省成本并提高性能。

Stylus: 扩散模型的自动适配器选择

Stylus: Automatic Adapter Selection for Diffusion Models

Low-Rank Adaptation (LoRA) has recently gained attention for fine-tuning
foundation models by incorporating trainable low-rank matrices, thereby
reducing the number of trainable parameters. While LoRA offers numerous
advantages, its applicability for real-time serving to a diverse and global
user base is constrained by its incapability to handle multiple task-specific
adapters efficiently. This imposes a performance bottleneck in scenarios
requiring personalized, task-specific adaptations for each incoming request. To
mitigate this constraint, we introduce Fast LoRA (FLoRA), a framework in which
each input example in a minibatch can be associated with its unique low-rank
adaptation weights, allowing for efficient batching of heterogeneous requests.
We empirically demonstrate that FLoRA retains the performance merits of LoRA,
showcasing competitive results on the MultiPL-E code generation benchmark
spanning over 8 languages and a multilingual speech recognition task across 6
languages.

通过引入名为 Fast LoRA（FLoRA）的框架，我们可以有效地对多样化和全球用户群体的实时请求进行批处理，通过将每个输入示例与其独特的低秩适应权重关联起来，实现个性化的任务特定适应，从而缓解了 Low-Rank Adaptation (LoRA) 在处理多个任务特定适配器时的性能瓶颈。我们在包括 8 种语言的 MultiPL-E 代码生成基准和 6 种语言的多语种语音识别任务上，通过实证展示了 FLoRA 保持 LoRA 性能优点的竞争结果。