Parameter-Efficient Fine-Tuning (PEFT) is increasingly recognized as an effective method in speech processing. However, the optimal approach and the placement of PEFT methods remain inconclusive. Our study conducts extensive experiments to compare different PEFT methods and their layer-wise placement adapting Differentiable Architecture Search (DARTS). We also explore the use of ensemble learning to leverage diverse PEFT strategies. The results reveal that DARTS does not outperform the baseline approach, which involves inserting the same PEFT method into all layers of a Self-Supervised Learning (SSL) model. In contrast, an ensemble learning approach, particularly one employing majority voting, demonstrates superior performance. Our statistical evidence indicates that different PEFT methods learn in varied ways. This variation might explain why the synergistic integration of various PEFT methods through ensemble learning can harness their unique learning capabilities more effectively compared to individual layer-wise optimization.

通过在不同层次中插入不同的Parameter-Efficient Fine-Tuning方法并采用Differentiable Architecture Search (DARTS)进行比较，我们研究了PEFT方法的最佳途径和放置方式；同时，我们还探索了使用集成学习来提升PEFT策略的多样性。实验结果表明，DARTS并没有超越基准方法的表现，而将PEFT方法插入到自监督学习模型的所有层中的集成学习方法表现更佳。我们的统计证据表明，不同的PEFT方法以不同的方式进行学习，这种差异能够解释为什么通过集成学习的协同融合比单独进行层级优化更有效地利用它们的独特学习能力。

语音的PEFT: 揭示最佳部署、合并策略和集成技术