We introduce RecurrentGemma, an open language model which uses Google's novel
Griffin architecture. Griffin combines linear recurrences with local attention
to achieve excellent performance on language. It has a fixed-sized state, which
reduces memory use and enables efficient inference on long sequences. We
provide a pre-trained model with 2B non-embedding parameters, and an
instruction tuned variant. Both models achieve comparable performance to
Gemma-2B despite being trained on fewer tokens.

我们介绍了一个开放的语言模型，使用了谷歌的全新的 Griffin 架构。Griffin 结合了线性递归和局部注意力，在语言方面有出色的表现。它具有固定大小的状态，可减少内存使用并在长序列上实现高效的推理。我们提供了一个已经预训练的模型，其中包含 2B 个非嵌入参数，还有经过调整的指令变种。尽管这两个模型的训练标记较少，但它们都实现了与 Gemma-2B 相当的性能。