Inspired by the success of large language models (LLM) for DNA and proteins, several LLM for RNA have been developed recently. RNA-LLM uses large datasets of RNA sequences to learn, in a self-supervised way, how to represent each RNA base with a semantically rich numerical vector. This is done under the hypothesis that obtaining high-quality RNA representations can enhance data-costly downstream tasks. Among them, predicting the secondary structure is a fundamental task for uncovering RNA functional mechanisms. In this work we present a comprehensive experimental analysis of several pre-trained RNA-LLM, comparing them for the RNA secondary structure prediction task in an unified deep learning framework. The RNA-LLM were assessed with increasing generalization difficulty on benchmark datasets. Results showed that two LLM clearly outperform the other models, and revealed significant challenges for generalization in low-homology scenarios.

本研究旨在解决当前RNA二级结构预测中的模型性能差异问题。通过在统一深度学习框架下对多种预训练RNA语言模型进行全面实验分析，揭示了在低同源性场景中面临的重大挑战。研究发现，有两个模型在预测精度上显著优于其他模型，展示了更高质量RNA表征在预测任务中的潜在影响。

大型语言模型在RNA二级结构预测中的全面基准测试