Field experiments and computer simulations are effective but time-consuming
methods of measuring the quality of engineered systems at different settings.
To reduce the total time required, experimenters may employ Bayesian
optimization, which is parsimonious with measurements, and take measurements of
multiple settings simultaneously, in a batch. In practice, experimenters use
very few batches, thus, it is imperative that each batch be as informative as
possible. Typically, the initial batch in a Batch Bayesian Optimization (BBO)
is constructed from a quasi-random sample of settings values. We propose a
batch-design acquisition function, Minimal Terminal Variance (MTV), that
designs a batch by optimization rather than random sampling. MTV adapts a
design criterion function from Design of Experiments, called I-Optimality,
which minimizes the variance of the post-evaluation estimates of quality,
integrated over the entire space of settings. MTV weights the integral by the
probability that a setting is optimal, making it able to design not only an
initial batch but all subsequent batches, as well. Applicability to both
initialization and subsequent batches is novel among acquisition functions.
Numerical experiments on test functions and simulators show that MTV compares
favorably to other BBO methods.

采用 Bayesian optimization 的 Minimal Terminal Variance (MTV) 方法通过优化设计而非随机抽样选择实验批次，并应用于初始批次和后续批次，以减少设计和评估工程系统质量所需的总时间。

批次贝叶斯优化的最佳初始化

Optimal Initialization of Batch Bayesian Optimization

In recent years, generative AI has undergone major advancements,
demonstrating significant promise in augmenting human productivity. Notably,
large language models (LLM), with ChatGPT-4 as an example, have drawn
considerable attention. Numerous articles have examined the impact of LLM-based
tools on human productivity in lab settings and designed tasks or in
observational studies. Despite recent advances, field experiments applying
LLM-based tools in realistic settings are limited. This paper presents the
findings of a field randomized controlled trial assessing the effectiveness of
LLM-based tools in providing unmonitored support services for information
retrieval.

通过一个领域性的随机控制试验，评估基于大型语言模型的工具在提供无监控支持服务方面的有效性。

聊天机器人的人工智能革命：来自一项随机对照实验的证明

AI Revolution on Chat Bot: Evidence from a Randomized Controlled  Experiment

Online experiments are widely used to compare specific design alternatives,
but they can also be used to produce generalizable knowledge and inform
strategic decision making. Doing so often requires sophisticated experimental
designs, iterative refinement, and careful logging and analysis. Few tools
exist that support these needs. We thus introduce a language for online field
experiments called PlanOut. PlanOut separates experimental design from
application code, allowing the experimenter to concisely describe experimental
designs, whether common "A/B tests" and factorial designs, or more complex
designs involving conditional logic or multiple experimental units. These
latter designs are often useful for understanding causal mechanisms involved in
user behaviors. We demonstrate how experiments from the literature can be
implemented in PlanOut, and describe two large field experiments conducted on
Facebook with PlanOut. For common scenarios in which experiments are run
iteratively and in parallel, we introduce a namespaced management system that
encourages sound experimental practice.

介绍了一种用于在线实验的语言 PlanOut，支持常见和更复杂的实验设计，可用于理解用户行为的因果机制，通过两个在 Facebook 上进行的大型实验展示其应用。

在线场地试验的设计和部署

Designing and Deploying Online Field Experiments

Social advertising uses information about consumers' peers, including peer
affiliations with a brand, product, organization, etc., to target ads and
contextualize their display. This approach can increase ad efficacy for two
main reasons: peers' affiliations reflect unobserved consumer characteristics,
which are correlated along the social network; and the inclusion of social cues
(i.e., peers' association with a brand) alongside ads affect responses via
social influence processes. For these reasons, responses may be increased when
multiple social signals are presented with ads, and when ads are affiliated
with peers who are strong, rather than weak, ties.
We conduct two very large field experiments that identify the effect of
social cues on consumer responses to ads, measured in terms of ad clicks and
the formation of connections with the advertised entity. In the first
experiment, we randomize the number of social cues present in word-of-mouth
advertising, and measure how responses increase as a function of the number of
cues. The second experiment examines the effect of augmenting traditional ad
units with a minimal social cue (i.e., displaying a peer's affiliation below an
ad in light grey text). On average, this cue causes significant increases in ad
performance. Using a measurement of tie strength based on the total amount of
communication between subjects and their peers, we show that these influence
effects are greatest for strong ties. Our work has implications for ad
optimization, user interface design, and central questions in social science
research.

社交广告利用消费者同伴的信息，以及同伴与品牌、产品、组织等的从属关系，来定向广告和增加其有效性。该研究通过两项实地实验发现，多个社交信号与广告一起展示，以及与强联系人相关联的广告，可以显著增加消费者对广告的反应。