Large Language Models (LLMs) have demonstrated remarkable capabilities across
various domains and are moving towards more specialized areas. Recent advanced
proprietary models such as GPT-4 and Gemini have achieved significant
advancements in biomedicine, which have also raised privacy and security
challenges. The construction of specialized generalists hinges largely on
high-quality datasets, enhanced by techniques like supervised fine-tuning and
reinforcement learning from human or AI feedback, and direct preference
optimization. However, these leading technologies (e.g., preference learning)
are still significantly limited in the open source community due to the
scarcity of specialized data. In this paper, we present the UltraMedical
collections, which consist of high-quality manual and synthetic datasets in the
biomedicine domain, featuring preference annotations across multiple advanced
LLMs. By utilizing these datasets, we fine-tune a suite of specialized medical
models based on Llama-3 series, demonstrating breathtaking capabilities across
various medical benchmarks. Moreover, we develop powerful reward models skilled
in biomedical and general reward benchmark, enhancing further online preference
learning within the biomedical LLM community.

利用高质量手动和合成数据集，本研究在生物医学领域构建了一套专用医学模型，基于 Llama-3 系列进行了精细调优，展示了在各种医学基准测试上的令人惊叹的能力。同时，我们还开发了强大的擅长生物医学和一般奖励基准的奖励模型，进一步增强了生物医学 LLM 社区内的在线优先学习能力。