mRNA-based vaccines have become a major focus in the pharmaceutical industry. The coding sequence as well as the Untranslated Regions (UTRs) of an mRNA can strongly influence translation efficiency, stability, degradation, and other factors that collectively determine a vaccine's effectiveness. However, optimizing mRNA sequences for those properties remains a complex challenge. Existing deep learning models often focus solely on coding region optimization, overlooking the UTRs. We present Helix-mRNA, a structured state-space-based and attention hybrid model to address these challenges. In addition to a first pre-training, a second pre-training stage allows us to specialise the model with high-quality data. We employ single nucleotide tokenization of mRNA sequences with codon separation, ensuring prior biological and structural information from the original mRNA sequence is not lost. Our model, Helix-mRNA, outperforms existing methods in analysing both UTRs and coding region properties. It can process sequences 6x longer than current approaches while using only 10% of the parameters of existing foundation models. Its predictive capabilities extend to all mRNA regions. We open-source the model (https://github.com/helicalAI/helical) and model weights (https://huggingface.co/helical-ai/helix-mRNA).

本研究针对当前mRNA序列优化中的复杂挑战，提出了Helix-mRNA模型，该模型能够同时分析编码区和非翻译区（UTRs）。通过两阶段预训练与单核苷酸标记化，Helix-mRNA在处理更长序列的同时，仅使用现有基础模型的10%的参数，实现了优于现有方法的预测性能，对疫苗效果的改进具有重要潜力。

Helix-mRNA：一种用于全序列mRNA治疗的混合基础模型