High-quality instruction data is critical for aligning large language models
(LLMs). Although some models, such as Llama-3-Instruct, have open weights,
their alignment data remain private, which hinders the democratization of AI.
High human labor costs and a limited, predefined scope for prompting prevent
existing open-source data creation methods from scaling effectively,
potentially limiting the diversity and quality of public alignment datasets. Is
it possible to synthesize high-quality instruction data at scale by extracting
it directly from an aligned LLM? We present a self-synthesis method for
generating large-scale alignment data named Magpie. Our key observation is that
aligned LLMs like Llama-3-Instruct can generate a user query when we input only
the left-side templates up to the position reserved for user messages, thanks
to their auto-regressive nature. We use this method to prompt Llama-3-Instruct
and generate 4 million instructions along with their corresponding responses.
We perform a comprehensive analysis of the extracted data and select 300K
high-quality instances. To compare Magpie data with other public instruction
datasets, we fine-tune Llama-3-8B-Base with each dataset and evaluate the
performance of the fine-tuned models. Our results indicate that in some tasks,
models fine-tuned with Magpie perform comparably to the official
Llama-3-8B-Instruct, despite the latter being enhanced with 10 million data
points through supervised fine-tuning (SFT) and subsequent feedback learning.
We also show that using Magpie solely for SFT can surpass the performance of
previous public datasets utilized for both SFT and preference optimization,
such as direct preference optimization with UltraFeedback. This advantage is
evident on alignment benchmarks such as AlpacaEval, ArenaHard, and WildBench.

通过自我综合的方法，从对齐的大型语言模型中直接提取高质量的指令数据，称为 Magpie，以生成大规模的对齐数据

Magpie: 无助推主题、从头训练，通过提示对齐的 LLMs 生成对齐数据

Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs  with Nothing

Media bias detection poses a complex, multifaceted problem traditionally
tackled using single-task models and small in-domain datasets, consequently
lacking generalizability. To address this, we introduce MAGPIE, the first
large-scale multi-task pre-training approach explicitly tailored for media bias
detection. To enable pre-training at scale, we present Large Bias Mixture
(LBM), a compilation of 59 bias-related tasks. MAGPIE outperforms previous
approaches in media bias detection on the Bias Annotation By Experts (BABE)
dataset, with a relative improvement of 3.3% F1-score. MAGPIE also performs
better than previous models on 5 out of 8 tasks in the Media Bias
Identification Benchmark (MBIB). Using a RoBERTa encoder, MAGPIE needs only 15%
of finetuning steps compared to single-task approaches. Our evaluation shows,
for instance, that tasks like sentiment and emotionality boost all learning,
all tasks enhance fake news detection, and scaling tasks leads to the best
results. MAGPIE confirms that MTL is a promising approach for addressing media
bias detection, enhancing the accuracy and efficiency of existing models.
Furthermore, LBM is the first available resource collection focused on media
bias MTL.

MAGPIE 是第一个专门针对媒体偏见检测的大规模多任务预训练方法，通过提出 Large Bias Mixture (LBM)，编制了 59 个与偏见相关的任务。实验证明，MAGPIE 在 Bias Annotation By Experts (BABE) 数据集上的媒体偏见检测表现优于之前的方法，相对提升了 3.3% 的 F1 得分。MAGPIE 在 Media Bias Identification Benchmark (MBIB) 的 8 个任务中有 5 个任务表现更好。通过使用 RoBERTa 编码器，相比单任务方法，MAGPIE 只需要 15% 的微调步骤。结果显示，情感和情绪等任务可提升所有任务的学习，所有任务都可以增强对假新闻的检测，而扩展任务可以取得最佳结果。MAGPIE 证实了多任务学习是解决媒体偏见检测的一种有前景的方法，提高了现有模型的准确性和效率。此外，LBM 是第一个专注于媒体偏见多任务学习的资源收集。