There has been a surge in the development of various Large Language Models
(LLMs). However, text generation for languages other than English often faces
significant challenges, including poor generation quality and the reduced
computational performance due to the disproportionate representation of tokens
in model's vocabulary. In this work, we address these issues and introduce
Vikhr, a new state-of-the-art open-source instruction-tuned LLM designed
specifically for the Russian language. Unlike previous efforts for Russian that
utilize computationally inexpensive LoRA adapters on top of English-oriented
models, Vikhr features an adapted tokenizer vocabulary and undergoes the
continued pre-training and instruction tuning of all weights. This approach not
only enhances the model's performance but also significantly improves its
computational and contextual efficiency. The remarkable performance of Vikhr
across various Russian-language benchmarks can also be attributed to our
efforts in expanding instruction datasets and corpora for continued
pre-training. Vikhr not only sets the new state of the art among open-source
LLMs for Russian, but even outperforms some proprietary closed-source models on
certain benchmarks. The model weights, instruction sets, and code are publicly
available

为了解决非英语文本生成的挑战，如生成质量差和计算性能下降等问题，本研究介绍了一种专为俄语设计的开源指令调整大型语言模型 Vikhr，通过适应性分词词汇表、持续预训练和指令调整权重等方法，提高模型性能和计算效率，并在俄语基准测试中取得显著成果。