Large Language Models (LLMs) are prone to generating content that exhibits
gender biases, raising significant ethical concerns. Alignment, the process of
fine-tuning LLMs to better align with desired behaviors, is recognized as an
effective approach to mitigate gender biases. Although proprietary LLMs have
made significant strides in mitigating gender bias, their alignment datasets
are not publicly available. The commonly used and publicly available alignment
dataset, HH-RLHF, still exhibits gender bias to some extent. There is a lack of
publicly available alignment datasets specifically designed to address gender
bias. Hence, we developed a new dataset named GenderAlign, aiming at mitigating
a comprehensive set of gender biases in LLMs. This dataset comprises 8k
single-turn dialogues, each paired with a "chosen" and a "rejected" response.
Compared to the "rejected" responses, the "chosen" responses demonstrate lower
levels of gender bias and higher quality. Furthermore, we categorized the
gender biases in the "rejected" responses of GenderAlign into 4 principal
categories. The experimental results show the effectiveness of GenderAlign in
reducing gender bias in LLMs.

通过开发名为 GenderAlign 的数据集，本研究旨在减少大型语言模型中多个维度的性别偏见，并将性别偏见分类为 4 个主要类别。实验证明 GenderAlign 在减轻大型语言模型的性别偏见方面具有显著效果。

GenderAlign：用于减弱大型语言模型中性别偏差的对齐数据集

GenderAlign: An Alignment Dataset for Mitigating Gender Bias in Large  Language Models

Effectively handling instructions with extremely long context remains a
challenge for Large Language Models (LLMs), typically necessitating
high-quality long data and substantial computational resources. This paper
introduces Step-Skipping Alignment (SkipAlign), a new technique designed to
enhance the long-context capabilities of LLMs in the phase of alignment without
the need for additional efforts beyond training with original data length.
SkipAlign is developed on the premise that long-range dependencies are
fundamental to enhancing an LLM's capacity of long context. Departing from
merely expanding the length of input samples, SkipAlign synthesizes long-range
dependencies from the aspect of positions indices. This is achieved by the
strategic insertion of skipped positions within instruction-following samples,
which utilizes the semantic structure of the data to effectively expand the
context. Through extensive experiments on base models with a variety of context
window sizes, SkipAlign demonstrates its effectiveness across a spectrum of
long-context tasks. Particularly noteworthy is that with a careful selection of
the base model and alignment datasets, SkipAlign with only 6B parameters
achieves it's best performance and comparable with strong baselines like
GPT-3.5-Turbo-16K on LongBench.

该研究论文介绍了一种名为 Step-Skipping Alignment 的技术，为大型语言模型（LLMs）增强了长上下文能力，通过在指令 - 跟随样本中策略性地插入跳过的位置来合成长程依赖，从而有效扩展上下文，并通过在不同上下文窗口大小的基础模型上进行广泛实验来验证其有效性。