Language Models (LMs) excel in understanding textual descriptions of
proteins, as evident in biomedical question-answering tasks. However, their
capability falters with raw protein data, such as amino acid sequences, due to
a deficit in pretraining on such data. Conversely, Protein Language Models
(PLMs) can understand and convert protein data into high-quality
representations, but struggle to process texts. To address their limitations,
we introduce ProtT3, a framework for Protein-to-Text Generation for Text-based
Protein Understanding. ProtT3 empowers an LM to understand protein sequences of
amino acids by incorporating a PLM as its protein understanding module,
enabling effective protein-to-text generation. This collaboration between PLM
and LM is facilitated by a cross-modal projector (i.e., Q-Former) that bridges
the modality gap between the PLM's representation space and the LM's input
space. Unlike previous studies focusing on protein property prediction and
protein-text retrieval, we delve into the largely unexplored field of
protein-to-text generation. To facilitate comprehensive benchmarks and promote
future research, we establish quantitative evaluations for protein-text
modeling tasks, including protein captioning, protein question-answering, and
protein-text retrieval. Our experiments show that ProtT3 substantially
surpasses current baselines, with ablation studies further highlighting the
efficacy of its core components. Our code is available at
this https URL

ProtT3 是一个框架，通过将 Protein Language Models 作为其蛋白质理解模块，使语言模型能够理解氨基酸序列，并实现有效的蛋白质到文本生成。

ProtT3: 基于文本的蛋白质理解的蛋白质到文本生成

ProtT3: Protein-to-Text Generation for Text-based Protein Understanding

Language Models (LMs) have demonstrated impressive molecule understanding
ability on various 1D text-related tasks. However, they inherently lack 2D
graph perception - a critical ability of human professionals in comprehending
molecules' topological structures. To bridge this gap, we propose MolCA:
Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal
Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and
graph-based molecular contents via the cross-modal projector. Specifically, the
cross-modal projector is implemented as a Q-Former to connect a graph encoder's
representation space and an LM's text space. Further, MolCA employs a uni-modal
adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks.
Unlike previous studies that couple an LM with a graph encoder via cross-modal
contrastive learning, MolCA retains the LM's ability of open-ended text
generation and augments it with 2D graph information. To showcase its
effectiveness, we extensively benchmark MolCA on tasks of molecule captioning,
IUPAC name prediction, and molecule-text retrieval, on which MolCA
significantly outperforms the baselines. Our codes and checkpoints can be found
at this https URL

MolCA 是一种使用跨模态计算器和单模适配器的分子图 - 语言建模方法，能够通过将图编码器的表示空间和语言模型的文本空间连接，使语言模型理解基于文本和图的分子内容。