The existing fine-tuning paradigm for language models is brittle in knowledge
editing scenarios, where the model must incorporate new information without
extensive retraining. This brittleness often results in overfitting, reduced
performance, and unnatural language generation. To address this, we propose
Consistent In-Context Editing (ICE), a novel approach that leverages the
model's in-context learning capability to tune toward a contextual distribution
rather than a one-hot target. ICE introduces a straightforward optimization
framework that includes both a target and a procedure, enhancing the robustness
and effectiveness of gradient-based tuning methods. We provide analytical
insights into ICE across four critical aspects of knowledge editing: accuracy,
locality, generalization, and linguistic quality, showing its advantages.
Experimental results across four datasets confirm the effectiveness of ICE and
demonstrate its potential for continual editing, ensuring that updated
information is incorporated while preserving the integrity of the model.

已有的语言模型微调范式在知识编辑场景下容易出现脆弱性，本文提出了一种名为一致上下文编辑（ICE）的新方法，利用模型的上下文学习能力来调整模型以适应上下文分布而非单一目标，通过直观的优化框架增强了梯度调整方法的鲁棒性和效果，实验证明了 ICE 的优势及其在不断编辑中的潜力，确保了更新的信息被纳入同时保持模型的完整性。

上下文编辑：从自引导分布中学习知识

In-Context Editing: Learning Knowledge from Self-Induced Distributions

Recent trends in Large Vision Language Models (LVLMs) research have been
increasingly focusing on advancing beyond general image understanding towards
more nuanced, object-level referential comprehension. In this paper, we present
and delve into the self-consistency capability of LVLMs, a crucial aspect that
reflects the models' ability to both generate informative captions for specific
objects and subsequently utilize these captions to accurately re-identify the
objects in a closed-loop process. This capability significantly mirrors the
precision and reliability of fine-grained visual-language understanding. Our
findings reveal that the self-consistency level of existing LVLMs falls short
of expectations, posing limitations on their practical applicability and
potential. To address this gap, we introduce a novel fine-tuning paradigm named
Self-Consistency Tuning (SC-Tune). It features the synergistic learning of a
cyclic describer-locator system. This paradigm is not only data-efficient but
also exhibits generalizability across multiple LVLMs. Through extensive
experiments, we demonstrate that SC-Tune significantly elevates performance
across a spectrum of object-level vision-language benchmarks and maintains
competitive or improved performance on image-level vision-language benchmarks.
Both our model and code will be publicly available at
this https URL

我们介绍了 LVLM 的自一致能力和新颖的自一致调整范式 (SC-Tune)，证明了 SC-Tune 显著提升了目标级别视觉语言基准的性能，并在图像级视觉语言基准上保持了竞争力或改进的性能。

SC-Tune：解锁大型视觉语言模型中的自洽参考理解

SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large  Vision Language Models

Existing Neural Machine Translation (NMT) models mainly handle translation in
the general domain, while overlooking domains with special writing formulas,
such as e-commerce and legal documents. Taking e-commerce as an example, the
texts usually include amounts of domain-related words and have more grammar
problems, which leads to inferior performances of current NMT methods. To
address these problems, we collect two domain-related resources, including a
set of term pairs (aligned Chinese-English bilingual terms) and a parallel
corpus annotated for the e-commerce domain. Furthermore, we propose a two-step
fine-tuning paradigm (named G2ST) with self-contrastive semantic enhancement to
transfer one general NMT model to the specialized NMT model for e-commerce. The
paradigm can be used for the NMT models based on Large language models (LLMs).
Extensive evaluations on real e-commerce titles demonstrate the superior
translation quality and robustness of our G2ST approach, as compared with
state-of-the-art NMT models such as LLaMA, Qwen, GPT-3.5, and even GPT-4.

利用两个领域相关资源，提出了一个两步细调范式来将一般 NMT 模型转化为专门用于电商领域的 NMT 模型，该范式基于大型语言模型（LLMs），通过自对比语义增强来提高翻译质量和鲁棒性。与最先进的 NMT 模型（如 LLaMA、Qwen、GPT-3.5 甚至 GPT-4）相比，对真实电商标题进行的广泛评估表明，我们的 G2ST 方法具有卓越的翻译质量和鲁棒性。