The recently proposed generative flow networks (GFlowNets) are a method of
training a policy to sample compositional discrete objects with probabilities
proportional to a given reward via a sequence of actions. GFlowNets exploit the
sequential nature of the problem, drawing parallels with reinforcement learning
(RL). Our work extends the connection between RL and GFlowNets to a general
case. We demonstrate how the task of learning a generative flow network can be
efficiently redefined as an entropy-regularized RL problem with a specific
reward and regularizer structure. Furthermore, we illustrate the practical
efficiency of this reformulation by applying standard soft RL algorithms to
GFlowNet training across several probabilistic modeling tasks. Contrary to
previously reported results, we show that entropic RL approaches can be
competitive against established GFlowNet training methods. This perspective
opens a direct path for integrating reinforcement learning principles into the
realm of generative flow networks.

最近提出的生成流网络（GFlowNets）是一种通过一系列操作，使策略从概率与给定奖励成比例的组合离散对象中采样的方法。我们的工作将强化学习和 GFlowNets 的联系扩展到一般情况，演示了如何将学习生成流网络的任务高效重新定义为具有特定奖励和正则化结构的熵正则化强化学习问题，并通过在几个概率建模任务上应用标准的软强化学习算法展示了这种改进的实际效率。与先前报告的结果相反，我们表明熵正则化强化学习方法可以与已建立的 GFlowNet 训练方法竞争。这一观点为将强化学习原理整合到生成流网络领域开辟了直接道路。