This work introduces Weaver, our first family of large language models (LLMs)
dedicated to content creation. Weaver is pre-trained on a carefully selected
corpus that focuses on improving the writing capabilities of large language
models. We then fine-tune Weaver for creative and professional writing purposes
and align it to the preference of professional writers using a suit of novel
methods for instruction data synthesis and LLM alignment, making it able to
produce more human-like texts and follow more diverse instructions for content
creation. The Weaver family consists of models of Weaver Mini (1.8B), Weaver
Base (6B), Weaver Pro (14B), and Weaver Ultra (34B) sizes, suitable for
different applications and can be dynamically dispatched by a routing agent
according to query complexity to balance response quality and computation cost.
Evaluation on a carefully curated benchmark for assessing the writing
capabilities of LLMs shows Weaver models of all sizes outperform generalist
LLMs several times larger than them. Notably, our most-capable Weaver Ultra
model surpasses GPT-4, a state-of-the-art generalist LLM, on various writing
scenarios, demonstrating the advantage of training specialized LLMs for writing
purposes. Moreover, Weaver natively supports retrieval-augmented generation
(RAG) and function calling (tool usage). We present various use cases of these
abilities for improving AI-assisted writing systems, including integration of
external knowledge bases, tools, or APIs, and providing personalized writing
assistance. Furthermore, we discuss and summarize a guideline and best
practices for pre-training and fine-tuning domain-specific LLMs.

Weaver 是一种专注于内容创作的大型语言模型（LLM）系列，通过精心挑选的语料库进行预训练，再通过创造性和专业写作目的进行微调，使其能够生成更接近人类风格的文本并满足更多样化的内容创作指令。Weaver 系列各个模型规模适用于不同应用，并且可以根据查询复杂性进行动态分配以平衡响应质量和计算成本。评估表明，相较于比它们规模大数倍的综合型 LLM，包括最强的 Weaver Ultra 模型也优于 GPT-4，展示了为写作目的进行专门训练的 LLM 的优势。此外，Weaver 还原生支持检索增强生成（RAG）和函数调用（工具使用），为改进 AI 辅助写作系统提供各种用例和最佳实践指南。