With excellent generalization ability, self-supervised speech models have
shown impressive performance on various downstream speech tasks in the
pre-training and fine-tuning paradigm. However, as the growing size of
pre-trained models, fine-tuning becomes practically unfeasible due to heavy
computation and storage overhead, as well as the risk of overfitting. Adapters
are lightweight modules inserted into pre-trained models to facilitate
parameter-efficient adaptation. In this paper, we propose an effective adapter
framework designed for adapting self-supervised speech models to the speaker
verification task. With a parallel adapter design, our proposed framework
inserts two types of adapters into the pre-trained model, allowing the
adaptation of latent features within intermediate Transformer layers and output
embeddings from all Transformer layers. We conduct comprehensive experiments to
validate the efficiency and effectiveness of the proposed framework.
Experimental results on the VoxCeleb1 dataset demonstrate that the proposed
adapters surpass fine-tuning and other parameter-efficient transfer learning
methods, achieving superior performance while updating only 5% of the
parameters.

通过在预先训练的模型中插入轻量级适配器模块，将自监督语音模型适应到说话人验证任务中，我们提出了一个高效的适配器框架。实验结果表明，该框架在更新仅 5% 的参数的情况下，超过了微调和其他参数高效的迁移学习方法，实现了卓越的性能。

预训练语音模型的高效适配器调优用于自动说话人验证

Efficient Adapter Tuning of Pre-trained Speech Models for Automatic  Speaker Verification

In recent years, "pre-training and fine-tuning" has emerged as a promising
approach in addressing the issues of label dependency and poor generalization
performance in traditional GNNs. To reduce labeling requirement, the
"pre-train, fine-tune" and "pre-train, prompt" paradigms have become
increasingly common. In particular, prompt tuning is a popular alternative to
"pre-training and fine-tuning" in natural language processing, which is
designed to narrow the gap between pre-training and downstream objectives.
However, existing study of prompting on graphs is still limited, lacking a
framework that can accommodate commonly used graph pre-training methods and
downstream tasks. In this paper, we propose a multi-view graph contrastive
learning method as pretext and design a prompting tuning for it. Specifically,
we first reformulate graph pre-training and downstream tasks into a common
format. Second, we construct multi-view contrasts to capture relevant
information of graphs by GNN. Third, we design a prompting tuning method for
our multi-view graph contrastive learning method to bridge the gap between
pretexts and downsteam tasks. Finally, we conduct extensive experiments on
benchmark datasets to evaluate and analyze our proposed method.

我们提出了一种多视角图对比学习方法，并为其设计了提示调整方法，以缩小预训练和下游任务之间的差距。

多视图图对比学习的提示调优

Prompt Tuning for Multi-View Graph Contrastive Learning

The "pre-training and fine-tuning" paradigm in addressing long-tailed
recognition tasks has sparked significant interest since the emergence of large
vision-language models like the contrastive language-image pre-training (CLIP).
While previous studies have shown promise in adapting pre-trained models for
these tasks, they often undesirably require extensive training epochs or
additional training data to maintain good performance. In this paper, we
propose PEL, a fine-tuning method that can effectively adapt pre-trained models
to long-tailed recognition tasks in fewer than 20 epochs without the need for
extra data. We first empirically find that commonly used fine-tuning methods,
such as full fine-tuning and classifier fine-tuning, suffer from overfitting,
resulting in performance deterioration on tail classes. To mitigate this issue,
PEL introduces a small number of task-specific parameters by adopting the
design of any existing parameter-efficient fine-tuning method. Additionally, to
expedite convergence, PEL presents a novel semantic-aware classifier
initialization technique derived from the CLIP textual encoder without adding
any computational overhead. Our experimental results on four long-tailed
datasets demonstrate that PEL consistently outperforms previous
state-of-the-art approaches. The source code is available at
this https URL

通过引入 PEL 方法，该研究通过少于 20 个时期的微调，无需额外数据即可适应长尾识别任务，并通过在分类器初始化中采用 CLIP 文本编码器的新颖技术解决了过度拟合问题，从而持续优于之前的最佳方法。