We introduce Llama Guard, an LLM-based input-output safeguard model geared
towards Human-AI conversation use cases. Our model incorporates a safety risk
taxonomy, a valuable tool for categorizing a specific set of safety risks found
in LLM prompts (i.e., prompt classification). This taxonomy is also
instrumental in classifying the responses generated by LLMs to these prompts, a
process we refer to as response classification. For the purpose of both prompt
and response classification, we have meticulously gathered a dataset of high
quality. Llama Guard, a Llama2-7b model that is instruction-tuned on our
collected dataset, albeit low in volume, demonstrates strong performance on
existing benchmarks such as the OpenAI Moderation Evaluation dataset and
ToxicChat, where its performance matches or exceeds that of currently available
content moderation tools. Llama Guard functions as a language model, carrying
out multi-class classification and generating binary decision scores.
Furthermore, the instruction fine-tuning of Llama Guard allows for the
customization of tasks and the adaptation of output formats. This feature
enhances the model's capabilities, such as enabling the adjustment of taxonomy
categories to align with specific use cases, and facilitating zero-shot or
few-shot prompting with diverse taxonomies at the input. We are making Llama
Guard model weights available and we encourage researchers to further develop
and adapt them to meet the evolving needs of the community for AI safety.

我们引入了 Llama Guard，一种基于 LLM 的输入输出保护模型，针对人工智能与人类对话应用场景设计。我们的模型包含一种安全风险分类法，用于对 LLM 提示中一组特定安全风险进行分类（即提示分类）。此分类法还用于对 LLM 生成的响应进行分类（即响应分类）的过程。为了进行提示和响应的分类，我们精心收集了高质量的数据集。Llama Guard 是一个在我们收集的数据集上进行指令调整的 Llama2-7b 模型，尽管数据量较少，但在现有基准测试中表现出色，如 OpenAI 评估数据集和 ToxicChat，其表现与当前可用的内容审查工具相匹配或超过。Llama Guard 作为一种语言模型，执行多类别分类并生成二进制决策分数。此外，Llama Guard 的指令微调允许定制任务和调整输出格式。此功能增强了模型的能力，例如使得能够调整分类法类别以适应特定用例，并促进零射击或少射击提示与多样化的分类法输入的配合。我们提供 Llama Guard 模型权重，并鼓励研究人员进一步开发和调整，以满足人工智能安全社区不断发展的需求。