We present an end-to-end framework for generating synthetic users for evaluating interactive agents designed to encourage positive behavior changes, such as in health and lifestyle coaching. The synthetic users are grounded in health and lifestyle conditions, specifically sleep and diabetes management in this study, to ensure realistic interactions with the health coaching agent. Synthetic users are created in two stages: first, structured data are generated grounded in real-world health and lifestyle factors in addition to basic demographics and behavioral attributes; second, full profiles of the synthetic users are developed conditioned on the structured data. Interactions between synthetic users and the coaching agent are simulated using generative agent-based models such as Concordia, or directly by prompting a language model. Using two independently-developed agents for sleep and diabetes coaching as case studies, the validity of this framework is demonstrated by analyzing the coaching agent's understanding of the synthetic users' needs and challenges. Finally, through multiple blinded evaluations of user-coach interactions by human experts, we demonstrate that our synthetic users with health and behavioral attributes more accurately portray real human users with the same attributes, compared to generic synthetic users not grounded in such attributes. The proposed framework lays the foundation for efficient development of conversational agents through extensive, realistic, and grounded simulated interactions.

本研究针对健康和生活方式辅导中缺乏真实用户交互的问题，提出了一种端到端的合成用户生成框架，特别关注睡眠和糖尿病管理。利用真实世界健康因素生成合成用户的结构化数据，并发展完整用户档案，使得与辅导代理的互动更为真实有效。通过多个盲评实验，结果显示合成用户在健康和行为属性上的表现比通用合成用户更能真实反映拥有相同属性的人类用户。

不眠之夜，甜蜜之日：为真实辅导代理互动创建具有健康状况的合成用户