In this work we introduce RITA: a suite of autoregressive generative models
for protein sequences, with up to 1.2 billion parameters, trained on over 280
million protein sequences belonging to the UniRef-100 database. Such generative
models hold the promise of greatly accelerating protein design. We conduct the
first systematic study of how capabilities evolve with model size for
autoregressive transformers in the protein domain: we evaluate RITA models in
next amino acid prediction, zero-shot fitness, and enzyme function prediction,
showing benefits from increased scale. We release the RITA models openly, to
the benefit of the research community.

本文提出了 RITA: 一套自回归的生成模型，用于蛋白质序列，具有高达 12 亿个参数，训练了超过 2.8 亿个属于 UniRef-100 数据库的蛋白质序列。我们首次系统性研究了自回归变压器在蛋白质领域中的能力随着模型大小的发展，我们评估了 RITA 模型在接下来的氨基酸预测，零样本适应性以及酶功能预测中的表现，并展示了规模扩大的好处。我们公开发布了 RITA 模型，以造福研究社区。