The recent advancement of pre-trained Transformer models has propelled the
development of effective text mining models across various biomedical tasks.
However, these models are primarily learned on the textual data and often lack
the domain knowledge of the entities to capture the context beyond the
sentence. In this study, we introduced a novel framework that enables the model
to learn multi-omnics biological information about entities (proteins) with the
help of additional multi-modal cues like molecular structure. Towards this,
rather developing modality-specific architectures, we devise a generalized and
optimized graph based multi-modal learning mechanism that utilizes the
GraphBERT model to encode the textual and molecular structure information and
exploit the underlying features of various modalities to enable end-to-end
learning. We evaluated our proposed method on ProteinProtein Interaction task
from the biomedical corpus, where our proposed generalized approach is observed
to be benefited by the additional domain-specific modality.

本研究介绍了一种使用多模态线索（如分子结构）帮助模型学习实体（蛋白质）的多组学生物信息的新颖框架，以图形为基础的多模态学习机制利用 GraphBERT 模型编码了文本和分子结构信息，并利用不同模态的基础特征进行端到端的学习。在生物医学领域的蛋白质相互作用任务中，我们的提出的广义方法发现受到了领域特定模态的额外获益。