Protein design involves generating protein sequences based on their
corresponding protein backbones. While deep generative models show promise for
learning protein design directly from data, the lack of publicly available
structure-sequence pairings limits their generalization capabilities. Previous
efforts of generative protein design have focused on architectural improvements
and pseudo-data augmentation to overcome this bottleneck. To further address
this challenge, we propose a novel protein design paradigm called MMDesign,
which leverages multi-modality transfer learning. To our knowledge, MMDesign is
the first framework that combines a pretrained structural module with a
pretrained contextual module, using an auto-encoder (AE) based language model
to incorporate prior semantic knowledge of protein sequences. We also introduce
a cross-layer cross-modal alignment algorithm to enable the structural module
to learn long-term temporal information and ensure consistency between
structural and contextual modalities. Experimental results, only training with
the small CATH dataset, demonstrate that our MMDesign framework consistently
outperforms other baselines on various public test sets. To further assess the
biological plausibility of the generated protein sequences and data
distribution, we present systematic quantitative analysis techniques that
provide interpretability and reveal more about the laws of protein design.

MMDesign 是一个结合了先前结构模块和语境模块的预训练模型的新型蛋白质设计范式，利用自编码器（AE）语言模型将蛋白质序列的先验语义知识纳入其中，并引入了一种跨层跨模态对齐算法来确保结构模块学习到长期时间信息并保持结构和语境模态的一致性，在仅使用小规模 CATH 数据集进行训练的实验结果表明，我们的 MMDesign 框架在各种公共测试集上始终优于其他基线方法，并提出了系统性定量分析技术来评估生成的蛋白质序列和数据分布的生物可能性，以展示其可解释性并揭示蛋白质设计的规律。