We develop Scalable Latent Exploration Score (ScaLES) to mitigate
over-exploration in Latent Space Optimization (LSO), a popular method for
solving black-box discrete optimization problems. LSO utilizes continuous
optimization within the latent space of a Variational Autoencoder (VAE) and is
known to be susceptible to over-exploration, which manifests in unrealistic
solutions that reduce its practicality. ScaLES is an exact and theoretically
motivated method leveraging the trained decoder's approximation of the data
distribution. ScaLES can be calculated with any existing decoder, e.g. from a
VAE, without additional training, architectural changes, or access to the
training data. Our evaluation across five LSO benchmark tasks and three VAE
architectures demonstrates that ScaLES enhances the quality of the solutions
while maintaining high objective values, leading to improvements over existing
solutions. We believe that new avenues to LSO will be opened by ScaLES ability
to identify out of distribution areas, differentiability, and computational
tractability. Open source code for ScaLES is available at
this https URL

我们开发了可扩展的潜在探索评分（ScaLES）来减轻潜在空间优化中的过度探索问题，通过利用训练解码器对数据分布的近似来进行计算，ScaLES 提高了解决方案的质量，同时保持较高的客观值，从而改进了现有的解决方案。

ScaLES：大规模潜在探索分数用于预训练生成网络

ScaLES: Scalable Latent Exploration Score for Pre-Trained Generative  Networks

This paper considers a stochastic multi-armed bandit (MAB) problem with dual
objectives: (i) quick identification and commitment to the optimal arm, and
(ii) reward maximization throughout a sequence of $T$ consecutive rounds.
Though each objective has been individually well-studied, i.e., best arm
identification for (i) and regret minimization for (ii), the simultaneous
realization of both objectives remains an open problem, despite its practical
importance. This paper introduces \emph{Regret Optimal Best Arm Identification}
(ROBAI) which aims to achieve these dual objectives. To solve ROBAI with both
pre-determined stopping time and adaptive stopping time requirements, we
present the $\mathsf{EOCP}$ algorithm and its variants respectively, which not
only achieve asymptotic optimal regret in both Gaussian and general bandits,
but also commit to the optimal arm in $\mathcal{O}(\log T)$ rounds with
pre-determined stopping time and $\mathcal{O}(\log^2 T)$ rounds with adaptive
stopping time. We further characterize lower bounds on the commitment time
(equivalent to sample complexity) of ROBAI, showing that $\mathsf{EOCP}$ and
its variants are sample optimal with pre-determined stopping time, and almost
sample optimal with adaptive stopping time. Numerical results confirm our
theoretical analysis and reveal an interesting ``over-exploration'' phenomenon
carried by classic $\mathsf{UCB}$ algorithms, such that $\mathsf{EOCP}$ has
smaller regret even though it stops exploration much earlier than
$\mathsf{UCB}$ ($\mathcal{O}(\log T)$ versus $\mathcal{O}(T)$), which suggests
over-exploration is unnecessary and potentially harmful to system performance.

通过介绍一种新算法 ROBAI 和其变种，该研究识别并解决了在带有双重目标的多臂赌博机问题中达到最优臂的同时最大化奖励的难题；并对算法的停止时间、样本复杂性以及与经典 UCB 算法相比的性能进行了理论分析和数值实验，揭示出了经典 UCB 算法中的 “过度探索” 现象。