In the face of rapidly accumulating genomic data, our understanding of the
rna regulatory code remains incomplete. Recent self-supervised methods in other
domains have demonstrated the ability to learn rules underlying the
data-generating process such as sentence structure in language.