Effective DNA embedding remains crucial in genomic analysis, particularly in
scenarios lacking labeled data for model fine-tuning, despite the significant
advancements in genome foundation models. A prime example is metagenomics
binning, a critical process in microbiome research that aims to group DNA
sequences by their species from a complex mixture of DNA sequences derived from
potentially thousands of distinct, often uncharacterized species. To fill the
lack of effective DNA embedding models, we introduce DNABERT-S, a genome
foundation model that specializes in creating species-aware DNA embeddings. To
encourage effective embeddings to error-prone long-read DNA sequences, we
introduce Manifold Instance Mixup (MI-Mix), a contrastive objective that mixes
the hidden representations of DNA sequences at randomly selected layers and
trains the model to recognize and differentiate these mixed proportions at the
output layer. We further enhance it with the proposed Curriculum Contrastive
Learning (C$^2$LR) strategy. Empirical results on 18 diverse datasets showed
DNABERT-S's remarkable performance. It outperforms the top baseline's
performance in 10-shot species classification with just a 2-shot training while
doubling the Adjusted Rand Index (ARI) in species clustering and substantially
increasing the number of correctly identified species in metagenomics binning.
The code, data, and pre-trained model are publicly available at
this https URL

通过引入 DNABERT-S 模型对 DNA 进行嵌入，结合 Manifold Instance Mixup 和 Curriculum Contrastive Learning（C2LR）策略，本研究在基因组学分析中取得了显著的性能提升，尤其在元基因组分组和物种分类方面表现突出。