Communication is supposed to improve multi-agent collaboration and overall
performance in cooperative Multi-agent reinforcement learning (MARL). However,
such improvements are prevalently limited in practice since most existing
communication schemes ignore communication overheads (e.g., communication
delays). In this paper, we demonstrate that ignoring communication delays has
detrimental effects on collaborations, especially in delay-sensitive tasks such
as autonomous driving. To mitigate this impact, we design a delay-aware
multi-agent communication model (DACOM) to adapt communication to delays.
Specifically, DACOM introduces a component, TimeNet, that is responsible for
adjusting the waiting time of an agent to receive messages from other agents
such that the uncertainty associated with delay can be addressed. Our
experiments reveal that DACOM has a non-negligible performance improvement over
other mechanisms by making a better trade-off between the benefits of
communication and the costs of waiting for messages.

本研究提出了一个针对多智能体强化学习的通信模型，通过引入 TimeNet 来解决通信延迟问题，从而在自动驾驶等延迟敏感型任务中提高了协作性能。

DACOM：面向多智能体强化学习的时延感知沟通学习

DACOM: Learning Delay-Aware Communication for Multi-Agent Reinforcement  Learning

Inspired by the tremendous success of deep Convolutional Neural Networks as
generic feature extractors for images, we propose TimeNet: a deep recurrent
neural network (RNN) trained on diverse time series in an unsupervised manner
using sequence to sequence (seq2seq) models to extract features from time
series. Rather than relying on data from the problem domain, TimeNet attempts
to generalize time series representation across domains by ingesting time
series from several domains simultaneously. Once trained, TimeNet can be used
as a generic off-the-shelf feature extractor for time series. The
representations or embeddings given by a pre-trained TimeNet are found to be
useful for time series classification (TSC). For several publicly available
datasets from UCR TSC Archive and an industrial telematics sensor data from
vehicles, we observe that a classifier learned over the TimeNet embeddings
yields significantly better performance compared to (i) a classifier learned
over the embeddings given by a domain-specific RNN, as well as (ii) a nearest
neighbor classifier based on Dynamic Time Warping.

TimeNet 是一个基于循环神经网络的深度非监督学习模型，使用序列到序列模型从多个时间序列中提取特征，可用作时间序列的通用特征提取器，并且通过实验证明，使用 TimeNet 特征提取器训练的分类器能够显著提高分类效果。