Categorical random variables can faithfully represent the discrete and
uncertain aspects of data as part of a discrete latent variable model. Learning
in such models necessitates taking gradients with respect to the parameters of
the categorical probability distributions, which is often intractable due to
their combinatorial nature. A popular technique to estimate these otherwise
intractable gradients is the Log-Derivative trick. This trick forms the basis
of the well-known REINFORCE gradient estimator and its many extensions. While
the Log-Derivative trick allows us to differentiate through samples drawn from
categorical distributions, it does not take into account the discrete nature of
the distribution itself. Our first contribution addresses this shortcoming by
introducing the CatLog-Derivative trick - a variation of the Log-Derivative
trick tailored towards categorical distributions. Secondly, we use the
CatLog-Derivative trick to introduce IndeCateR, a novel and unbiased gradient
estimator for the important case of products of independent categorical
distributions with provably lower variance than REINFORCE. Thirdly, we
empirically show that IndeCateR can be efficiently implemented and that its
gradient estimates have significantly lower bias and variance for the same
number of samples compared to the state of the art.

我们提出了 CatLog-Derivative 技巧，该技巧针对分类分布，并用于引入 IndeCateR，一个新颖且无偏的梯度估计器，具有比 REINFORCE 更低方差的重要独立分类分布产品情况，并通过实验证明同样数量的样本下 IndeCateR 的梯度估计具有显著较低的偏差和方差。