Generative Flow Networks (GFlowNets) are amortized sampling methods for
learning a stochastic policy to sequentially generate compositional objects
with probabilities proportional to their rewards. GFlowNets exhibit a
remarkable ability to generate diverse sets of high-reward objects, in contrast
to standard return maximization reinforcement learning approaches, which often
converge to a single optimal solution. Recent works have arisen for learning
goal-conditioned GFlowNets to acquire various useful properties, aiming to
train a single GFlowNet capable of achieving different goals as the task
specifies. However, training a goal-conditioned GFlowNet poses critical
challenges due to extremely sparse rewards, which is further exacerbated in
large state spaces. In this work, we propose a novel method named Retrospective
Backward Synthesis (RBS) to address these challenges. Specifically, RBS
synthesizes a new backward trajectory based on the backward policy in GFlowNets
to enrich training trajectories with enhanced quality and diversity, thereby
efficiently solving the sparse reward problem. Extensive empirical results show
that our method improves sample efficiency by a large margin and outperforms
strong baselines on various standard evaluation benchmarks.

通过回顾性逆向合成（RBS）方法，我们提出一种应对稀疏奖励问题的新方法，用于训练目标条件下的生成流网络（GFlowNets），并在各类标准评估基准上显著提高样本效率并优于强基准模型。