While general-purpose large language models (LLMs) demonstrate proficiency on
multiple tasks within the domain of translation, approaches based on open LLMs
are competitive only when specializing on a single task. In this paper, we
propose a recipe for tailoring LLMs to multiple tasks present in translation
workflows. We perform continued pretraining on a multilingual mixture of
monolingual and parallel data, creating TowerBase, followed by finetuning on
instructions relevant for translation processes, creating TowerInstruct. Our
final model surpasses open alternatives on several tasks relevant to
translation workflows and is competitive with general-purpose closed LLMs. To
facilitate future research, we release the Tower models, our specialization
dataset, an evaluation framework for LLMs focusing on the translation
ecosystem, and a collection of model generations, including ours, on our
benchmark.

我们提出了一种多任务定制通用大型语言模型的方法，通过在多语言混合单语和平行数据上进行持续预训练，创建了 TowerBase，并在与翻译过程相关的指令上进行微调，创建了 TowerInstruct。我们的最终模型在与翻译工作流相关的多个任务上超过了开放式替代方案，并与通用封闭式语言模型相媲美。为了促进未来研究，我们发布了 Tower 模型、我们的专业数据集、一个关注翻译生态系统的 LLM 评估框架以及一系列模型生成版本的基准。