We present $\textbf{Platypus}$, a family of fine-tuned and merged Large
Language Models (LLMs) that achieves the strongest performance and currently
stands at first place in HuggingFace's Open LLM Leaderboard as of the release
date of this work. In this work we describe (1) our curated dataset
$\textbf{Open-Platypus}$, that is a subset of other open datasets and which
$\textit{we release to the public}$ (2) our process of fine-tuning and merging
LoRA modules in order to conserve the strong prior of pretrained LLMs, while
bringing specific domain knowledge to the surface (3) our efforts in checking
for test data leaks and contamination in the training data, which can inform
future research. Specifically, the Platypus family achieves strong performance
in quantitative LLM metrics across model sizes, topping the global Open LLM
leaderboard while using just a fraction of the fine-tuning data and overall
compute that are required for other state-of-the-art fine-tuned LLMs. In
particular, a 13B Platypus model can be trained on $\textit{a single}$ A100 GPU
using 25k questions in 5 hours. This is a testament of the quality of our
Open-Platypus dataset, and opens opportunities for more improvements in the
field. Project page: this https URL

我们提出的 $	extbf {Platypus}$ 家族的大型语言模型在 HuggingFace 的 Open LLM Leaderboard 上表现最好，并且在本研究发布日期时处于首位。我们描述了我们的精选数据集 $	extbf {Open-Platypus}$，它是其他开放数据集的子集，我们向公众发布。我们的工作包括微调和合并 LoRA 模块以保留预训练 LLM 的强先验，并展现特定领域的知识，以及检查训练数据中的测试数据泄漏和污染的努力，这可以为未来的研究提供信息。特别是，Platypus 家族在各种模型大小的定量 LLM 指标上表现出色，在使用其它最先进的微调 LLM 所需的微调数据和计算资源的一小部分的情况下，超过全球 Open LLM leaderboard。特别是 13B 的 Platypus 模型可以在单个 A100 GPU 上训练，在 5 小时内使用 25k 个问题。这证明了我们 Open-Platypus 数据集的质量，为该领域的更多改进开辟了机会。