Phased releases are a common strategy in the technology industry for
gradually releasing new products or updates through a sequence of A/B tests in
which the number of treated units gradually grows until full deployment or
deprecation. Performing phased releases in a principled way requires selecting
the proportion of units assigned to the new release in a way that balances the
risk of an adverse effect with the need to iterate and learn from the
experiment rapidly. In this paper, we formalize this problem and propose an
algorithm that automatically determines the release percentage at each stage in
the schedule, balancing the need to control risk while maximizing ramp-up
speed. Our framework models the challenge as a constrained batched bandit
problem that ensures that our pre-specified experimental budget is not depleted
with high probability. Our proposed algorithm leverages an adaptive Bayesian
approach in which the maximal number of units assigned to the treatment is
determined by the posterior distribution, ensuring that the probability of
depleting the remaining budget is low. Notably, our approach analytically
solves the ramp sizes by inverting probability bounds, eliminating the need for
challenging rare-event Monte Carlo simulation. It only requires computing means
and variances of outcome subsets, making it highly efficient and
parallelizable.

这篇论文提出了通过一系列的 A/B 实验和自适应 Bayesian 方法来保证控制风险和最大化增长速度的策略，该方法被称为 batched bandit 问题，其中关键因素是实验预算。