BriefGPT.xyz
Mar, 2017
REBAR: 离散潜变量模型低方差、无偏梯度估计
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
HTML
PDF
George Tucker, Andriy Mnih, Chris J. Maddison, Jascha Sohl-Dickstein
TL;DR
本文通过将控制变量与连续松弛相结合的方式来降低离散潜在变量的高方差梯度估计，并引入了一种在线调整松弛度的修改方法，实现了最先进的方差降低并加速了生成建模任务中的收敛。
Abstract
Learning in models with
discrete latent variables
is challenging due to high variance gradient estimators. Generally, approaches have relied on
control variates
to reduce the variance of the REINFORCE estimator.
→