General-purpose Large Language Models (LLMs) like GPT-4 have achieved
remarkable advancements in machine translation (MT) by leveraging extensive web
content. On the other hand, translation-specific LLMs are built by pre-training
on domain-specific monolingual corpora and fine-tuning with human-annotated
translation data. Despite the superior performance, these methods either demand
an unprecedented scale of computing and data or substantial human editing and
annotation efforts. In this paper, we develop Ladder, a novel model-agnostic
and cost-effective tool to refine the performance of general LLMs for MT.
Ladder is trained on pseudo-refinement triplets which can be easily obtained
from existing LLMs without additional human cost. During training, we propose a
hierarchical fine-tuning strategy with an easy-to-hard schema, improving
Ladder's refining performance progressively. The trained Ladder can be
seamlessly integrated with any general-purpose LLMs to boost their translation
performance. By utilizing Gemma-2B/7B as the backbone, Ladder-2B can elevate
raw translations to the level of top-tier open-source models (e.g., refining
BigTranslate-13B with +6.91 BLEU and +3.52 COMET for XX-En), and Ladder-7B can
further enhance model performance to be on par with the state-of-the-art GPT-4.
Extensive ablation and analysis corroborate the effectiveness of Ladder in
diverse settings. Our code is available at this https URL

通过使用 Ladder，我们开发了一种新颖的与模型无关且成本有效的工具，用于改进用于机器翻译的通用 LLMs 的性能。通过在现有 LLMs 中获得的伪精炼三元组进行训练，Ladder 可以逐步改进其精炼性能，并与任何通用 LLMs 无缝集成，提升其翻译表现。

Ladder：将基于 LLM 的机器翻译推向更高层次的模型无关框架

Ladder: A Model-Agnostic Framework Boosting LLM-based Machine  Translation to the Next Level

We study a practical setting of continual learning: fine-tuning on a
pre-trained model continually. Previous work has found that, when training on
new tasks, the features (penultimate layer representations) of previous data
will change, called representational shift. Besides the shift of features, we
reveal that the intermediate layers' representational shift (IRS) also matters
since it disrupts batch normalization, which is another crucial cause of
catastrophic forgetting. Motivated by this, we propose ConFiT, a fine-tuning
method incorporating two components, cross-convolution batch normalization
(Xconv BN) and hierarchical fine-tuning. Xconv BN maintains pre-convolution
running means instead of post-convolution, and recovers post-convolution ones
before testing, which corrects the inaccurate estimates of means under IRS.
Hierarchical fine-tuning leverages a multi-stage strategy to fine-tune the
pre-trained network, preventing massive changes in Conv layers and thus
alleviating IRS. Experimental results on four datasets show that our method
remarkably outperforms several state-of-the-art methods with lower storage
overhead.

本文提出了一种包含两个组件（交叉卷积批量规范化和分层微调）的微调方法 ConFiT，旨在解决表示转移和批量规范化不一致的问题，并在四个数据集上得到了较低存储开销的优异性能表现。