It is imperative for Large language models (LLMs) to follow instructions with
elaborate requirements (i.e. Complex Instructions Following). Yet, it remains
under-explored how to enhance the ability of LLMs to follow complex
instructions with multiple constraints. To bridge the gap, we initially study
what training data is effective in enhancing complex constraints following
abilities. We found that training LLMs with instructions containing multiple
constraints enhances their understanding of complex instructions, especially
those with lower complexity levels. The improvement can even generalize to
compositions of out-of-domain constraints. Additionally, we further propose
methods addressing how to obtain and utilize the effective training data.
Finally, we conduct extensive experiments to prove the effectiveness of our
methods in terms of overall performance, training efficiency, and
generalization abilities under four settings.

使用包含多个约束条件的指令来训练大型语言模型，能提高其理解复杂指令，特别是对于低复杂性水平的指令，甚至可以推广到超出领域约束的组合，同时提出了获取和利用有效训练数据的方法，并通过广泛实验验证了方法在总体性能、训练效率和泛化能力方面的有效性。