The instruction-following ability of large language models enables humans to interact with AI agents in a natural way. However, when required to generate responses of a specific length, large language models often struggle to meet users' needs due to their inherent difficulty in accurately perceiving numerical constraints. To explore the ability of large language models to control the length of generated responses, we propose the Target Length Generation Task (TLG) and design two metrics, Precise Match (PM) and Flexible Match (FM) to evaluate the model's performance in adhering to specified response lengths. Furthermore, we introduce a novel, model-agnostic approach called Ruler, which employs Meta Length Tokens (MLTs) to enhance the instruction-following ability of large language models under length-constrained instructions. Specifically, Ruler equips LLMs with the ability to generate responses of a specified length based on length constraints within the instructions. Moreover, Ruler can automatically generate appropriate MLT when length constraints are not explicitly provided, demonstrating excellent versatility and generalization. Comprehensive experiments show the effectiveness of Ruler across different LLMs on Target Length Generation Task, e.g., at All Level 27.97 average gain on PM, 29.57 average gain on FM. In addition, we conduct extensive ablation experiments to further substantiate the efficacy and generalization of Ruler. Our code and data is available at https://github.com/Geaming2002/Ruler.

本研究针对大规模语言模型在生成指定长度响应时的困难，提出了目标长度生成任务（TLG）及评估模型执行力的精确匹配（PM）和灵活匹配（FM）指标. 我们引入了一种名为Ruler的模型无关方法，通过元长度标记（MLTs）增强模型的指令-following能力，能够在指定长度约束下生成响应，并在缺乏明确约束时自动生成合适的MLT. 实验表明Ruler在TLG上的效果显著，带来了明显的性能提升。

Ruler：一种与模型无关的控制大规模语言模型生成长度的方法