Practitioners conducting adaptive experiments often encounter two competing
priorities: reducing the cost of experimentation by effectively assigning
treatments during the experiment itself, and gathering information swiftly to
conclude the experiment and implement a treatment across the population.
Currently, the literature is divided, with studies on regret minimization
addressing the former priority in isolation, and research on best-arm
identification focusing solely on the latter. This paper proposes a unified
model that accounts for both within-experiment performance and post-experiment
outcomes. We then provide a sharp theory of optimal performance in large
populations that unifies canonical results in the literature. This unification
also uncovers novel insights. For example, the theory reveals that familiar
algorithms, like the recently proposed top-two Thompson sampling algorithm, can
be adapted to optimize a broad class of objectives by simply adjusting a single
scalar parameter. In addition, the theory reveals that enormous reductions in
experiment duration can sometimes be achieved with minimal impact on both
within-experiment and post-experiment regret.

本文提出了一个统一的模型，同时考虑了实验内表现和实验后结果，为大规模人群中的最佳表现提供了一个清晰的理论，并揭示了一些新的洞见。

优化自适应实验：遗憾最小化和最佳臂识别的统一方法

Optimizing Adaptive Experiments: A Unified Approach to Regret  Minimization and Best-Arm Identification

Typically, multi-armed bandit (MAB) experiments are analyzed at the end of
the study and thus require the analyst to specify a fixed sample size in
advance. However, in many online learning applications, it is advantageous to
continuously produce inference on the average treatment effect (ATE) between
arms as new data arrive and determine a data-driven stopping time for the
experiment. Existing work on continuous inference for adaptive experiments
assumes that the treatment assignment probabilities are bounded away from zero
and one, thus excluding nearly all standard bandit algorithms. In this work, we
develop the Mixture Adaptive Design (MAD), a new experimental design for
multi-armed bandits that enables continuous inference on the ATE with
guarantees on statistical validity and power for nearly any bandit algorithm.
On a high level, the MAD "mixes" a bandit algorithm of the user's choice with a
Bernoulli design through a tuning parameter $\delta_t$, where $\delta_t$ is a
deterministic sequence that controls the priority placed on the Bernoulli
design as the sample size grows. We show that for $\delta_t =
o\left(1/t^{1/4}\right)$, the MAD produces a confidence sequence that is
asymptotically valid and guaranteed to shrink around the true ATE. We
empirically show that the MAD improves the coverage and power of ATE inference
in MAB experiments without significant losses in finite-sample reward.

通过 Mixture Adaptive Design (MAD) 实验设计，与用户选择的多臂赌博算法结合 Bernoulli 设计，能够在几乎任何赌博算法中对平均处理效果 (ATE) 进行连续推论，保证统计有效性和功率，提高 MAB 实验中 ATE 推论的覆盖率和功效。

一种多臂赌博机的随时有效因果推断的实验设计

An Experimental Design for Anytime-Valid Causal Inference on Multi-Armed  Bandits

From scientific experiments to online A/B testing, the previously observed
data often affects how future experiments are performed, which in turn affects
which data will be collected. Such adaptivity introduces complex correlations
between the data and the collection procedure. In this paper, we prove that
when the data collection procedure satisfies natural conditions, then sample
means of the data have systematic \emph{negative} biases. As an example,
consider an adaptive clinical trial where additional data points are more
likely to be tested for treatments that show initial promise. Our surprising
result implies that the average observed treatment effects would underestimate
the true effects of each treatment. We quantitatively analyze the magnitude and
behavior of this negative bias in a variety of settings. We also propose a
novel debiasing algorithm based on selective inference techniques. In
experiments, our method can effectively reduce bias and estimation error.

通过实验和在线 A/B 测试，我们证明在数据收集程序满足自然条件时，样本数据有系统的负偏差。我们提出了一种基于选择性推理技术的新型去偏算法，可以在各种情况下有效地减少偏差和估计误差。