The 5' UTR, a regulatory region at the beginning of an mRNA molecule, plays a crucial role in regulating the translation process and impacts the protein expression level. Language models have showcased their effectiveness in decoding the functions of protein and genome sequences. Here, we introduced a language model for 5' UTR, which we refer to as the UTR-LM. The UTR-LM is pre-trained on endogenous 5' UTRs from multiple species and is further augmented with supervised information including secondary structure and minimum free energy. We fine-tuned the UTR-LM in a variety of downstream tasks. The model outperformed the best-known benchmark by up to 42% for predicting the Mean Ribosome Loading, and by up to 60% for predicting the Translation Efficiency and the mRNA Expression Level. The model also applies to identifying unannotated Internal Ribosome Entry Sites within the untranslated region and improves the AUPR from 0.37 to 0.52 compared to the best baseline. Further, we designed a library of 211 novel 5' UTRs with high predicted values of translation efficiency and evaluated them via a wet-lab assay. Experiment results confirmed that our top designs achieved a 32.5% increase in protein production level relative to well-established 5' UTR optimized for therapeutics.

我们引入了一个名为UTR-LM的5'UTR语言模型，该模型在多个物种的内源性5'UTR上进行了预训练，并使用了包括二级结构和最小自由能在内的监督信息进行了进一步增强，该模型在多个下游任务中表现优于已知的最佳基准，可用于预测核糖体的平均装载量以及翻译效率和mRNA表达水平，还可用于确定未注释的非编码区域内的内源性核糖体入站位点，并相对于最佳基准将AUPR从0.37提高到0.52。此外，我们设计了一个包含211个高预测翻译效率的译码区的库，并通过湿实验进行了评估，实验结果确认我们的顶级设计相对于为治疗学进行了优化的成熟5' UTR，可以获得32.5%的蛋白质产量增加。

一个解码mRNA的5'UTR语言模型和功能预测