Traffic signal control is an important problem in urban mobility with a
significant potential of economic and environmental impact. While there is a
growing interest in Reinforcement Learning (RL) for traffic signal control, the
work so far has focussed on learning through simulations which could lead to
inaccuracies due to simplifying assumptions. Instead, real experience data on
traffic is available and could be exploited at minimal costs. Recent progress
in offline or batch RL has enabled just that. Model-based offline RL methods,
in particular, have been shown to generalize from the experience data much
better than others.
We build a model-based learning framework which infers a Markov Decision
Process (MDP) from a dataset collected using a cyclic traffic signal control
policy that is both commonplace and easy to gather. The MDP is built with
pessimistic costs to manage out-of-distribution scenarios using an adaptive
shaping of rewards which is shown to provide better regularization compared to
the prior related work in addition to being PAC-optimal. Our model is evaluated
on a complex signalized roundabout showing that it is possible to build highly
performant traffic control policies in a data efficient manner.

本研究基于批量强化学习的方法，运用基于惩罚项的自适应奖励方式在普通的循环交通信号控制策略下，构建出一个马尔可夫决策过程（MDP）的学习框架，不仅提高了对于不同分布情境的管理优化，还显著提高了交通信号控制的效率。