The well-known Gumbel-Max trick for sampling from a categorical distribution
can be extended to sample $k$ elements without replacement. We show how to
implicitly apply this 'Gumbel-Top-$k$' trick on a factorized distribution over
sequences, allowing to draw exact samples without replacement using a
Stochastic Beam Search. Even for exponentially large domains, the number of
model evaluations grows only linear in $k$ and the maximum sampled sequence
length. The algorithm creates a theoretical connection between sampling and
(deterministic) beam search and can be used as a principled intermediate
alternative. In a translation task, the proposed method compares favourably
against alternatives to obtain diverse yet good quality translations. We show
that sequences sampled without replacement can be used to construct
low-variance estimators for expected sentence-level BLEU score and model
entropy.

应用 Gumbel-Top-k 技巧和分解可重复采样，使用随机束搜索进行无重复抽样序列模型的研究，发现序列采样中存在随机束搜索和确定性束搜索之间的理论联系，这一方法在翻译任务中表现优异，且采样无重复序列可用于构造期望 BLEU 得分和模型熵的低方差估计。

随机束和它们的发现：Gumbel-Top-k 技巧在不重复采样序列中的应用

Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for  Sampling Sequences Without Replacement

In this work, we show that discretizing action space for continuous control
is a simple yet powerful technique for on-policy optimization. The explosion in
the number of discrete actions can be efficiently addressed by a policy with
factorized distribution across action dimensions. We show that the discrete
policy achieves significant performance gains with state-of-the-art on-policy
optimization algorithms (PPO, TRPO, ACKTR) especially on high-dimensional tasks
with complex dynamics. Additionally, we show that an ordinal parameterization
of the discrete distribution can introduce the inductive bias that encodes the
natural ordering between discrete actions. This ordinal architecture further
significantly improves the performance of PPO/TRPO.

本文研究了对连续控制中动作空间的离散化对于基于策略优化的影响，发现动作空间的离散化采用可分解动作分布的策略可以有效地解决离散动作数量的爆炸性增长，并且在复杂动态高维任务上可以通过在策略中使用序数参数化引入自然排序从而获得性能显著提升的优越表现。