Predicting accurate future trajectories of multiple agents is essential for
autonomous systems, but is challenging due to the complex agent interaction and
the uncertainty in each agent's future behavior. Forecasting multi-agent
trajectories requires modeling two key dimensions: (1) time dimension, where we
model the influence of past agent states over future states; (2) social
dimension, where we model how the state of each agent affects others. Most
prior methods model these two dimensions separately, e.g., first using a
temporal model to summarize features over time for each agent independently and
then modeling the interaction of the summarized features with a social model.
This approach is suboptimal since independent feature encoding over either the
time or social dimension can result in a loss of information. Instead, we would
prefer a method that allows an agent's state at one time to directly affect
another agent's state at a future time. To this end, we propose a new
Transformer, AgentFormer, that jointly models the time and social dimensions.
The model leverages a sequence representation of multi-agent trajectories by
flattening trajectory features across time and agents. Since standard attention
operations disregard the agent identity of each element in the sequence,
AgentFormer uses a novel agent-aware attention mechanism that preserves agent
identities by attending to elements of the same agent differently than elements
of other agents. Based on AgentFormer, we propose a stochastic multi-agent
trajectory prediction model that can attend to features of any agent at any
previous timestep when inferring an agent's future position. The latent intent
of all agents is also jointly modeled, allowing the stochasticity in one
agent's behavior to affect other agents. Our method substantially improves the
state of the art on well-established pedestrian and autonomous driving
datasets.

论文提出了一个新的 Transformer 模型 AgentFormer，该模型可以联合模拟时间和社交两个维度，用于预测多智能体行为，同时考虑了多智能体的影响以及不确定性。该模型具有良好的性能，并在公共数据集上取得了 state-of-the-art 的结果。