We propose ProtLLM, a versatile cross-modal large language model (LLM) for
both protein-centric and protein-language tasks. ProtLLM features a unique
dynamic protein mounting mechanism, enabling it to handle complex inputs where
the natural language text is interspersed with an arbitrary number of proteins.
Besides, we propose the protein-as-word language modeling approach to train
ProtLLM. By developing a specialized protein vocabulary, we equip the model
with the capability to predict not just natural language but also proteins from
a vast pool of candidates. Additionally, we construct a large-scale interleaved
protein-text dataset, named InterPT, for pre-training. This dataset
comprehensively encompasses both (1) structured data sources like protein
annotations and (2) unstructured data sources like biological research papers,
thereby endowing ProtLLM with crucial knowledge for understanding proteins. We
evaluate ProtLLM on classic supervised protein-centric tasks and explore its
novel protein-language applications. Experimental results demonstrate that
ProtLLM not only achieves superior performance against protein-specialized
baselines on protein-centric tasks but also induces zero-shot and in-context
learning capabilities on protein-language tasks.

我们提出了 ProtLLM，这是一个通用的跨模态大语言模型，用于蛋白质中心和蛋白质语言任务。ProtLLM 具有独特的动态蛋白质装配机制，能够处理自然语言文本与任意数量的蛋白质交织在一起的复杂输入。此外，我们提出了基于蛋白质作为单词的语言建模方法来训练 ProtLLM。通过开发专门的蛋白质词汇表，我们使模型能够从大量候选项中预测不仅自然语言，还包括蛋白质。另外，我们构建了一个大规模的交织蛋白质 - 文本数据集，名为 InterPT，用于预训练。该数据集全面包含蛋白质注释等结构化数据源和生物研究论文等非结构化数据源，从而赋予 ProtLLM 理解蛋白质的重要知识。我们在经典的监督蛋白质中心任务上评估了 ProtLLM，并探索了其新颖的蛋白质语言应用。实验证明，ProtLLM 不仅在蛋白质中心任务上表现出卓越性能，并且在蛋白质语言任务上具有零 - shot 和上下文学习能力。