Large language models (LLMs), despite their breakthroughs on many challenging
benchmark tasks, lean to generate verbose responses and lack the
controllability of output complexity, which is usually preferred by human users
in practice. In this paper, we study how to precisely control multiple
linguistic complexities of LLM output by finetuning using off-the-shelf data.
To this end, we propose multi-control tuning (MCTune), which includes multiple
linguistic complexity values of ground-truth responses as controls in the input
for instruction tuning. We finetune LLaMA2-7B on Alpaca-GPT4 and WizardLM
datasets. Evaluations on widely used benchmarks demonstrate that our method
does not only improve LLMs' multi-complexity controllability substantially but
also retains or even enhances the quality of the responses as a side benefit.

通过细调用于现有数据的多控制调优方法，本文提出一种能够精确控制大语言模型输出的多个语言复杂性的方法。实验证明该方法不仅显著改善了大语言模型的多复杂性可控性，还保持甚至提升了响应质量。

大型语言模型的多目标语言控制

Multi-Objective Linguistic Control of Large Language Models

While Transformer-based neural machine translation (NMT) is very effective in
high-resource settings, many languages lack the necessary large parallel
corpora to benefit from it. In the context of low-resource (LR) MT between two
closely-related languages, a natural intuition is to seek benefits from
structural "shortcuts", such as copying subwords from the source to the target,
given that such language pairs often share a considerable number of identical
words, cognates, and borrowings. We test Pointer-Generator Networks for this
purpose for six language pairs over a variety of resource ranges, and find weak
improvements for most settings. However, analysis shows that the model does not
show greater improvements for closely-related vs. more distant language pairs,
or for lower resource ranges, and that the models do not exhibit the expected
usage of the mechanism for shared subwords. Our discussion of the reasons for
this behaviour highlights several general challenges for LR NMT, such as modern
tokenization strategies, noisy real-world conditions, and linguistic
complexities. We call for better scrutiny of linguistically motivated
improvements to NMT given the blackbox nature of Transformer models, as well as
for a focus on the above problems in the field.

在低资源机器翻译中，通过 Transformer 网络结构的结构性快捷方式，如从源语言复制子词到目标语言，可以为近缘语言对之间的机器翻译提供微弱的改进，但并不展现与远缘语言对或低资源范围下相似语言对之间更强的改进，并且模型对于共享子词的机制使用并不如预期，这突显了低资源机器翻译领域面临的诸多挑战，如现代分词策略、嘈杂的真实环境和语言复杂性，并呼吁对 Transformer 模型进行更好的语言学动机改进的审视及对该领域上述问题的关注。