This paper delves into the pressing need in Parameter-Efficient Fine-Tuning (PEFT) for Large Language Models (LLMs). While LLMs possess remarkable capabilities, their extensive parameter requirements and associated computational demands hinder their practicality and scalability for real-world applications. Our position paper highlights current states and the necessity of further studying into the topic, and recognizes significant challenges and open issues that must be addressed to fully harness the powerful abilities of LLMs. These challenges encompass novel efficient PEFT architectures, PEFT for different learning settings, PEFT combined with model compression techniques, and the exploration of PEFT for multi-modal LLMs. By presenting this position paper, we aim to stimulate further research and foster discussions surrounding more efficient and accessible PEFT for LLMs.

该研究论文从参数高效微调（PEFT）对大语言模型（LLMs）的迫切需求出发，强调了当前状态和进一步研究该主题的必要性，以及需要解决的重要挑战和开放问题，包括创新PEFT架构、不同学习设置的PEFT、结合模型压缩技术的PEFT以及多模态LLMs的PEFT探索。通过提出该立场论文，我们旨在激发进一步的研究，并促进围绕LLMs的更高效和可访问的PEFT的讨论。

面向更高参数效率的大语言模型微调：一篇立场论文