May, 2024
高稀疏性基础 Llama 模型的高效预训练和部署
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment
Abhinav Agarwalla, Abhay Gupta, Alexandre Marques, Shubhra Pandit, Michael Goin...
TL;DR通过稀疏性,我们能够以较小的模型实现更快的训练和推理加速,并且不牺牲准确性。