Collaborative perception allows each agent to enhance its perceptual
abilities by exchanging messages with others. It inherently results in a
trade-off between perception ability and communication costs. Previous works
transmit complete full-frame high-dimensional feature maps among agents,
resulting in substantial communication costs. To promote communication
efficiency, we propose only transmitting the information needed for the
collaborator's downstream task. This pragmatic communication strategy focuses
on three key aspects: i) pragmatic message selection, which selects
task-critical parts from the complete data, resulting in spatially and
temporally sparse feature vectors; ii) pragmatic message representation, which
achieves pragmatic approximation of high-dimensional feature vectors with a
task-adaptive dictionary, enabling communicating with integer indices; iii)
pragmatic collaborator selection, which identifies beneficial collaborators,
pruning unnecessary communication links. Following this strategy, we first
formulate a mathematical optimization framework for the
perception-communication trade-off and then propose PragComm, a multi-agent
collaborative perception system with two key components: i) single-agent
detection and tracking and ii) pragmatic collaboration. The proposed PragComm
promotes pragmatic communication and adapts to a wide range of communication
conditions. We evaluate PragComm for both collaborative 3D object detection and
tracking tasks in both real-world, V2V4Real, and simulation datasets, OPV2V and
V2X-SIM2.0. PragComm consistently outperforms previous methods with more than
32.7K times lower communication volume on OPV2V. Code is available at
github.com/PhyllisH/PragComm.

通过合作感知，我们提出一种多智能体合作的感知系统 PragComm，通过主动选择消息、优化消息表示和选择有益的合作者，从而降低通信成本并提高任务性能。