Can we simulate a sandbox society with generative agents to model human behavior, thereby reducing the over-reliance on real human trials for assessing public policies? In this work, we investigate the feasibility of simulating health-related decision-making, using vaccine hesitancy, defined as the delay in acceptance or refusal of vaccines despite the availability of vaccination services (MacDonald, 2015), as a case study. To this end, we introduce the VacSim framework with 100 generative agents powered by Large Language Models (LLMs). VacSim simulates vaccine policy outcomes with the following steps: 1) instantiate a population of agents with demographics based on census data; 2) connect the agents via a social network and model vaccine attitudes as a function of social dynamics and disease-related information; 3) design and evaluate various public health interventions aimed at mitigating vaccine hesitancy. To align with real-world results, we also introduce simulation warmup and attitude modulation to adjust agents' attitudes. We propose a series of evaluations to assess the reliability of various LLM simulations. Experiments indicate that models like Llama and Qwen can simulate aspects of human behavior but also highlight real-world alignment challenges, such as inconsistent responses with demographic profiles. This early exploration of LLM-driven simulations is not meant to serve as definitive policy guidance; instead, it serves as a call for action to examine social simulation for policy development.

本研究解决了在公共政策评估中对实际人类实验的过度依赖问题，通过疫苗犹豫的案例研究探讨使用生成代理模拟健康相关决策的可行性。提出的VacSim框架利用大型语言模型 powered 的100个代理模拟疫苗政策结果，实验结果显示这些模型能够模拟人类行为的某些方面，强调了现实世界对齐的挑战，并呼吁进一步探索社会模拟在政策制定中的应用。

生成代理社会能否模拟人类行为并为公共卫生政策提供信息？疫苗犹豫的案例研究