This paper introduces a novel framework for DNA sequence generation,
comprising two key components: DiscDiff, a Latent Diffusion Model (LDM)
tailored for generating discrete DNA sequences, and Absorb-Escape, a
post-training algorithm designed to refine these sequences. Absorb-Escape
enhances the realism of the generated sequences by correcting `round errors'
inherent in the conversion process between latent and input spaces. Our
approach not only sets new standards in DNA sequence generation but also
demonstrates superior performance over existing diffusion models, in generating
both short and long DNA sequences. Additionally, we introduce EPD-GenDNA, the
first comprehensive, multi-species dataset for DNA generation, encompassing
160,000 unique sequences from 15 species. We hope this study will advance the
generative modelling of DNA, with potential implications for gene therapy and
protein production.

该论文引入了一种用于 DNA 序列生成的新型框架，包括 DiscDiff，一种专门用于生成离散 DNA 序列的潜在扩散模型（LDM），和 Absorb-Escape，一种后期训练算法，用于优化这些序列。Absorb-Escape 通过纠正潜在和输入空间之间转换过程中固有的 ' 舍入误差 ' 来增强生成序列的真实性。我们的方法不仅在 DNA 序列生成方面树立了新的标准，而且在生成短 DNA 序列和长 DNA 序列方面表现出了优越性能。此外，我们还推出了 EPD-GenDNA，这是第一个综合性的多物种 DNA 生成数据集，包含来自 15 种物种的 160,000 个独特序列。我们希望这项研究能推动 DNA 的生成建模，对基因治疗和蛋白质生产可能产生影响。

DiscDiff：用于 DNA 序列生成的潜在扩散模型

DiscDiff: Latent Diffusion Model for DNA Sequence Generation

Discrete diffusion or flow models could enable faster and more controllable
sequence generation than autoregressive models. We show that na\"ive linear
flow matching on the simplex is insufficient toward this goal since it suffers
from discontinuities in the training target and further pathologies. To
overcome this, we develop Dirichlet flow matching on the simplex based on
mixtures of Dirichlet distributions as probability paths. In this framework, we
derive a connection between the mixtures' scores and the flow's vector field
that allows for classifier and classifier-free guidance. Further, we provide
distilled Dirichlet flow matching, which enables one-step sequence generation
with minimal performance hits, resulting in $O(L)$ speedups compared to
autoregressive models. On complex DNA sequence generation tasks, we demonstrate
superior performance compared to all baselines in distributional metrics and in
achieving desired design targets for generated sequences. Finally, we show that
our classifier-free guidance approach improves unconditional generation and is
effective for generating DNA that satisfies design targets. Code is available
at this https URL

离散扩散或流模型能够实现比自回归模型更快、更可控的序列生成。本文通过基于狄利克雷分布混合的概率路径，在平凡上的 na"ive 线性匹配流上开发出狄利克雷流匹配模型，以克服训练目标中的不连续性和进一步的病理问题，从而实现分类器或无分类器引导。此外，我们提供了精简的狄利克雷流匹配方法，实现了一步式序列生成，最小化性能损失，相对于自回归模型加速了 $O (L)$。在复杂的 DNA 序列生成任务中，我们展示了相对于基准模型在分布度量和实现设计目标上的优越性能。最后，我们证明了无分类器引导方法可以提高无条件生成，并且对满足设计目标的 DNA 生成有效。

狄利克雷流匹配及其在 DNA 序列设计中的应用

Dirichlet Flow Matching with Applications to DNA Sequence Design

The harnessing of machine learning, especially deep generative models, has
opened up promising avenues in the field of synthetic DNA sequence generation.
Whilst Generative Adversarial Networks (GANs) have gained traction for this
application, they often face issues such as limited sample diversity and mode
collapse. On the other hand, Diffusion Models are a promising new class of
generative models that are not burdened with these problems, enabling them to
reach the state-of-the-art in domains such as image generation. In light of
this, we propose a novel latent diffusion model, DiscDiff, tailored for
discrete DNA sequence generation. By simply embedding discrete DNA sequences
into a continuous latent space using an autoencoder, we are able to leverage
the powerful generative abilities of continuous diffusion models for the
generation of discrete data. Additionally, we introduce Fr\'echet
Reconstruction Distance (FReD) as a new metric to measure the sample quality of
DNA sequence generations. Our DiscDiff model demonstrates an ability to
generate synthetic DNA sequences that align closely with real DNA in terms of
Motif Distribution, Latent Embedding Distribution (FReD), and Chromatin
Profiles. Additionally, we contribute a comprehensive cross-species dataset of
150K unique promoter-gene sequences from 15 species, enriching resources for
future generative modelling in genomics. We will make our code public upon
publication.

通过深度生成模型，特别是深度扩散模型，来合成 DNA 序列在合成生物学领域开辟了新的前景。我们提出了一种新颖的潜在扩散模型 (DiscDiff)，通过将离散 DNA 序列嵌入连续的潜在空间，使用自编码器来利用连续扩散模型强大的生成能力生成离散数据。此外，我们引入了一种新的度量标准 Frechet Reconstruction Distance (FReD)，用于衡量 DNA 序列生成结果的样本质量。我们的 DiscDiff 模型能够生成与真实 DNA 序列在 Motif 分布、潜在嵌入分布 (FReD) 和染色质剖面方面紧密相符的合成 DNA 序列。此外，我们还贡献了一个包含 15 个物种的 15 万个唯一启动子 - 基因序列的全面跨物种数据集，为未来基因组学中的生成建模工作提供丰富的资源。我们将在发表后公开我们的代码。