Large Language Models (LLMs) have demonstrated impressive potential to
simulate human behavior. Using a causal inference framework, we empirically and
theoretically analyze the challenges of conducting LLM-simulated experiments,
and explore potential solutions. In the context of demand estimation, we show
that variations in the treatment included in the prompt (e.g., price of focal
product) can cause variations in unspecified confounding factors (e.g., price
of competitors, historical prices, outside temperature), introducing
endogeneity and yielding implausibly flat demand curves. We propose a
theoretical framework suggesting this endogeneity issue generalizes to other
contexts and won't be fully resolved by merely improving the training data.
Unlike real experiments where researchers assign pre-existing units across
conditions, LLMs simulate units based on the entire prompt, which includes the
description of the treatment. Therefore, due to associations in the training
data, the characteristics of individuals and environments simulated by the LLM
can be affected by the treatment assignment. We explore two potential
solutions. The first specifies all contextual variables that affect both
treatment and outcome, which we demonstrate to be challenging for a
general-purpose LLM. The second explicitly specifies the source of treatment
variation in the prompt given to the LLM (e.g., by informing the LLM that the
store is running an experiment). While this approach only allows the estimation
of a conditional average treatment effect that depends on the specific
experimental design, it provides valuable directional results for exploratory
analysis.

使用因果推断框架，我们在实证和理论上分析了进行 LLM 模拟实验的挑战，并探索了潜在解决方案，其中需求估计的上下文中，我们表明提示中所包含的处理变体（例如，商品价格）可能导致未指定混淆因素的变化（例如，竞争对手的价格，历史价格，外部温度），从而引入内生性并产生不太可能的平坦需求曲线。我们提出了一个理论框架，表明这种内生性问题推广到其他情境，并不能仅通过改进训练数据来完全解决。

利用语言模型模拟人类行为的挑战：因果推断视角

The Challenge of Using LLMs to Simulate Human Behavior: A Causal  Inference Perspective

Mobility service route design requires potential demand information to well
accommodate travel demand within the service region. Transit planners and
operators can access various data sources including household travel survey
data and mobile device location logs. However, when implementing a mobility
system with emerging technologies, estimating demand level becomes harder
because of more uncertainties with user behaviors. Therefore, this study
proposes an artificial intelligence-driven algorithm that combines sequential
transit network design with optimal learning. An operator gradually expands its
route system to avoid risks from inconsistency between designed routes and
actual travel demand. At the same time, observed information is archived to
update the knowledge that the operator currently uses. Three learning policies
are compared within the algorithm: multi-armed bandit, knowledge gradient, and
knowledge gradient with correlated beliefs. For validation, a new route system
is designed on an artificial network based on public use microdata areas in New
York City. Prior knowledge is reproduced from the regional household travel
survey data. The results suggest that exploration considering correlations can
achieve better performance compared to greedy choices in general. In future
work, the problem may incorporate more complexities such as demand elasticity
to travel time, no limitations to the number of transfers, and costs for
expansion.

本研究提出了一种结合顺序公交网络设计和最优学习的人工智能驱动算法，以逐步扩大路线系统并更新当前的操作员使用的知识，验证表明，考虑相关性的探索可以实现比贪婪选择更好的性能，在未来的研究中，该问题可能加入更多复杂性如出行时间弹性、换乘次数无限制和扩展成本等方面。

具有相关信念的最优学习顺序交通网络设计算法

A sequential transit network design algorithm with optimal learning  under correlated beliefs

The next generation of satellite constellations is designed to better address
the future needs of our connected society: highly-variable data demand, mobile
connectivity, and reaching more under-served regions. Artificial Intelligence
(AI) and learning-based methods are expected to become key players in the
industry, given the poor scalability and slow reaction time of current resource
allocation mechanisms. While AI frameworks have been validated for isolated
communication tasks or subproblems, there is still not a clear path to achieve
fully-autonomous satellite systems. Part of this issue results from the focus
on subproblems when designing models, instead of the necessary system-level
perspective. In this paper we try to bridge this gap by characterizing the
system-level needs that must be met to increase satellite autonomy, and
introduce three AI-based components (Demand Estimator, Offline Planner, and
Real Time Engine) that jointly address them. We first do a broad literature
review on the different subproblems and identify the missing links to the
system-level goals. In response to these gaps, we outline the three necessary
components and highlight their interactions. We also discuss how current models
can be incorporated into the framework and possible directions of future work.

本研究旨在通过确定增加卫星自主性所需的系统级需求，并引入三个基于人工智能的组件，共同解决它们，从而缩小问题所在的差距。这些组件包括需求估计器，离线计划器和实时引擎，它们的交互作用及如何将现有模型纳入框架和未来工作方向也在研究中讨论。