Training with mixed data distributions is a common and important part of
creating multi-task and instruction-following models. The diversity of the data
distributions and cost of joint training makes the optimization procedure
extremely challenging. Data mixing methods partially address this problem,
albeit having a sub-optimal performance across data sources and require
multiple expensive training runs. In this paper, we propose a simple and
efficient alternative for better optimization of the data sources by combining
models individually trained on each data source with the base model using basic
element-wise vector operations. The resulting model, namely Distribution Edited
Model (DEM), is 11x cheaper than standard data mixing and outperforms strong
baselines on a variety of benchmarks, yielding up to 6.2% improvement on MMLU,
11.5% on BBH, 16.1% on DROP, and 9.3% on HELM with models of size 3B to 13B.
Notably, DEM does not require full re-training when modifying a single
data-source, thus making it very flexible and scalable for training with
diverse data sources.

提出了一种简单和高效的方法，使用基本的逐元素向量操作将每个数据源上单独训练的模型与基本模型相结合，从而更好地优化数据源，得到一种名为分布编辑模型（DEM）的模型，与标准数据混合相比廉价 11 倍，并在各种基准测试中优于强基准，对大小为 3B 到 13B 的模型分别可以获得 MMLU 的 6.2％改进，BBH 的 11.5％改进，DROP 的 16.1％，HELM 的 9.3％改进。值得注意的是，DEM 在修改单个数据源时不需要完全重新训练，因此非常灵活和可扩展以适应各种数据源的训练。