Instruction tuning has recently been recognized as an effective way of
aligning Large Language Models (LLMs) to enhance their generalization ability
across various tasks. However, when tuning publicly accessible, centralized
LLMs with private instruction data, privacy concerns are inevitable. While
direct transfer of parameterized modules between models is a plausible approach
to address this, its implications and effectiveness need further exploration.
This paper focuses on Offsite-Tuning (OFT), a representative technique that
transfers transformer blocks between centralized LLMs and downstream emulators.
Given the limited understanding of the underlying mechanism of OFT, we perform
an empirical analysis on LLMs from the perspectives of representation and
functional similarity. Interestingly, our findings reveal a unique modular
structure within the layers of LLMs that appears to emerge as the model size
expands. Simultaneously, we note subtle but potentially significant changes in
representation and intermediate predictions across the layers. Inspired by
these observations, we propose CRaSh, involving Clustering, Removing, and
Sharing, a training-free strategy to derive improved emulators from LLMs. CRaSh
significantly boosts performance of OFT with billions of parameters.
Furthermore, we investigate the optimal solutions yielded by fine-tuning with
and without full model through the lens of loss landscape. Our findings
demonstrate a linear connectivity among these optima falling over the same
basin, thereby highlighting the effectiveness of CRaSh and OFT. The source code
is publicly available at this https URL

借助 CRaSh 方法，通过探索 Large Language Models (LLMs) 的模块化结构和表示相似性，本文研究了 Offsite-Tuning (OFT) 技术及其与后端模拟器之间的转换，提高其性能并解决隐私问题。研究发现模型大小会导致 LLMs 内层面的唯一模块结构的出现，并注意到隐含的表示和中间预测方面的微妙变化。实验结果证明 CRaSh 和 OFT 的有效性，且细调尝试的结构优化解具有线性连通性。

CRaSh：聚类、剪枝和共享增强细调，无需完整大型语言模型

CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without  Full Large Language Model

Transfer learning is important for foundation models to adapt to downstream
tasks. However, many foundation models are proprietary, so users must share
their data with model owners to fine-tune the models, which is costly and raise
privacy concerns. Moreover, fine-tuning large foundation models is
computation-intensive and impractical for most downstream users. In this paper,
we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning
framework that can adapt billion-parameter foundation models to downstream data
without access to the full model. In offsite-tuning, the model owner sends a
light-weight adapter and a lossy compressed emulator to the data owner, who
then fine-tunes the adapter on the downstream data with the emulator's
assistance. The fine-tuned adapter is then returned to the model owner, who
plugs it into the full model to create an adapted foundation model.
Offsite-tuning preserves both parties' privacy and is computationally more
efficient than the existing fine-tuning methods that require access to the full
model weights. We demonstrate the effectiveness of offsite-tuning on various
large language and vision foundation models. Offsite-tuning can achieve
comparable accuracy as full model fine-tuning while being privacy-preserving
and efficient, achieving 6.5x speedup and 5.6x memory reduction. Code is
available at this https URL.

本文提出一种名为 Offsite-Tuning 的隐私保护和高效的迁移学习框架，可以适应亿级基础模型到下游数据，同时保留各方的隐私，比现有的需要访问全部模型权重的微调方法在计算上更加高效。