In addressing the computational and memory demands of fine-tuning Large
Language Models(LLMs), we propose LoRA-SP(Streamlined Partial Parameter
Adaptation), a novel approach utilizing randomized half-selective parameter
freezing within the Low-Rank Adaptation(LoRA)framework. This method efficiently
balances pre-trained knowledge retention and adaptability for task-specific
optimizations. Through a randomized mechanism, LoRA-SP determines which
parameters to update or freeze, significantly reducing computational and memory
requirements without compromising model performance. We evaluated LoRA-SP
across several benchmark NLP tasks, demonstrating its ability to achieve
competitive performance with substantially lower resource consumption compared
to traditional full-parameter fine-tuning and other parameter-efficient
techniques. LoRA-SP innovative approach not only facilitates the deployment of
advanced NLP models in resource-limited settings but also opens new research
avenues into effective and efficient model adaptation strategies.

利用大型语言模型的计算和存储需求，我们提出了 LoRA-SP (简化的部分参数适应) 方法，它利用 LoRA 框架中的随机半选择性参数冻结来平衡预训练知识的保留和任务特定优化的适应性。通过随机机制，LoRA-SP 确定要更新或冻结的参数，显著减少计算和存储需求，同时不损害模型性能。我们在多个基准自然语言处理任务上评估了 LoRA-SP，展示了它在与传统的全参数微调和其他参数高效技术相比，以较低的资源消耗实现竞争性性能的能力。LoRA-SP 的创新方法不仅有助于在资源有限的环境中部署高级自然语言处理模型，还为有效和高效的模型适应策略开辟了新的研究途径。