Understanding the long-term impact of algorithmic interventions on society is
vital to achieving responsible AI. Traditional evaluation strategies often fall
short due to the complex, adaptive and dynamic nature of society. While
reinforcement learning (RL) can be a powerful approach for optimizing decisions
in dynamic settings, the difficulty of realistic environment design remains a
barrier to building robust agents that perform well in practical settings. To
address this issue we tap into the field of system dynamics (SD) as a
complementary method that incorporates collaborative simulation model
specification practices. We introduce SDGym, a low-code library built on the
OpenAI Gym framework which enables the generation of custom RL environments
based on SD simulation models. Through a feasibility study we validate that
well specified, rich RL environments can be generated from preexisting SD
models and a few lines of configuration code. We demonstrate the capabilities
of the SDGym environment using an SD model of the electric vehicle adoption
problem. We compare two SD simulators, PySD and BPTK-Py for parity, and train a
D4PG agent using the Acme framework to showcase learning and environment
interaction. Our preliminary findings underscore the dual potential of SD to
improve RL environment design and for RL to improve dynamic policy discovery
within SD models. By open-sourcing SDGym, the intent is to galvanize further
research and promote adoption across the SD and RL communities, thereby
catalyzing collaboration in this emerging interdisciplinary space.

通过引入系统动态方法（SD）作为一种补充方法，利用 SDGym 生成基于 SD 模拟模型的定制强化学习（RL）环境，验证了通过预先存在的 SD 模型和少量的配置代码可以生成良好指定、丰富的 RL 环境。