Generative modeling over discrete data has recently seen numerous success
stories, with applications spanning language modeling, biological sequence
design, and graph-structured molecular data. The predominant generative
modeling paradigm for discrete data is still autoregressive, with more recent
alternatives based on diffusion or flow-matching falling short of their
impressive performance in continuous data settings, such as image or video
generation. In this work, we introduce Fisher-Flow, a novel flow-matching model
for discrete data. Fisher-Flow takes a manifestly geometric perspective by
considering categorical distributions over discrete data as points residing on
a statistical manifold equipped with its natural Riemannian metric: the
$\textit{Fisher-Rao metric}$. As a result, we demonstrate discrete data itself
can be continuously reparameterised to points on the positive orthant of the
$d$-hypersphere $\mathbb{S}^d_+$, which allows us to define flows that map any
source distribution to target in a principled manner by transporting mass along
(closed-form) geodesics of $\mathbb{S}^d_+$. Furthermore, the learned flows in
Fisher-Flow can be further bootstrapped by leveraging Riemannian optimal
transport leading to improved training dynamics. We prove that the gradient
flow induced by Fisher-Flow is optimal in reducing the forward KL divergence.
We evaluate Fisher-Flow on an array of synthetic and diverse real-world
benchmarks, including designing DNA Promoter, and DNA Enhancer sequences.
Empirically, we find that Fisher-Flow improves over prior diffusion and
flow-matching models on these benchmarks.

Fisher-Flow 是一种用于离散数据生成建模的新型流匹配模型，采用几何学观点，将离散数据视为在统计流形上的点，通过转移沿 $d$- 超球面上的（闭合形式）测地线上的质量来定义流，可以优化训练动力学，提高性能，并在合成和实际基因序列设计等多个领域中优于传统扩散和匹配模型。