We propose an approach to simulating trajectories of multiple interacting
agents (road users) based on transformers and probabilistic graphical models
(PGMs), and apply it to the Waymo SimAgents challenge. The transformer baseline
is based on the MTR model, which predicts multiple future trajectories
conditioned on the past trajectories and static road layout features. We then
improve upon these generated trajectories using a PGM, which contains factors
which encode prior knowledge, such as a preference for smooth trajectories, and
avoidance of collisions with static obstacles and other moving agents. We
perform (approximate) MAP inference in this PGM using the Gauss-Newton method.
Finally we sample $K=32$ trajectories for each of the $N \sim 100$ agents for
the next $T=8 \Delta$ time steps, where $\Delta=10$ is the sampling rate per
second. Following the Model Predictive Control (MPC) paradigm, we only return
the first element of our forecasted trajectories at each step, and then we
replan, so that the simulation can constantly adapt to its changing
environment. We therefore call our approach "Model Predictive Simulation" or
MPS. We show that MPS improves upon the MTR baseline, especially in safety
critical metrics such as collision rate. Furthermore, our approach is
compatible with any underlying forecasting model, and does not require extra
training, so we believe it is a valuable contribution to the community.

我们提出了一种基于 transformers 和概率图模型的多个相互作用代理（道路用户）轨迹模拟方法，并将其应用于 Waymo SimAgents 挑战。我们的方法基于 MTR 模型，使用先验知识编码因子来生成并改进轨迹预测，该模型通过执行高斯牛顿方法的（近似）最大后验推理来采样数十个代理的未来轨迹。我们的方法名为 “模型预测模拟”（MPS），在安全关键指标（如碰撞率）方面改进了 MTR 基线，并且与任何潜在预测模型兼容，不需要额外的训练。

基于结构化图模型和变压器的模型预测仿真

Model Predictive Simulation Using Structured Graphical Models and  Transformers

Since reward functions are hard to specify, recent work has focused on
learning policies from human feedback. However, such approaches are impeded by
the expense of acquiring such feedback. Recent work proposed that agents have
access to a source of information that is effectively free: in any environment
that humans have acted in, the state will already be optimized for human
preferences, and thus an agent can extract information about what humans want
from the state. Such learning is possible in principle, but requires simulating
all possible past trajectories that could have led to the observed state. This
is feasible in gridworlds, but how do we scale it to complex tasks? In this
work, we show that by combining a learned feature encoder with learned inverse
models, we can enable agents to simulate human actions backwards in time to
infer what they must have done. The resulting algorithm is able to reproduce a
specific skill in MuJoCo environments given a single state sampled from the
optimal policy for that skill.

本文摘要：本研究的目的是基于人类反馈对智能体进行政策学习，同时通过学习特征编码器结合学习反向模型，从而使得智能体能够向后模拟人类行为以推断人类行为背后的动机。