Collaborative perception aims to mitigate the limitations of single-agent
perception, such as occlusions, by facilitating data exchange among multiple
agents. However, most current works consider a homogeneous scenario where all
agents use identity sensors and perception models. In reality, heterogeneous
agent types may continually emerge and inevitably face a domain gap when
collaborating with existing agents. In this paper, we introduce a new open
heterogeneous problem: how to accommodate continually emerging new
heterogeneous agent types into collaborative perception, while ensuring high
perception performance and low integration cost? To address this problem, we
propose HEterogeneous ALliance (HEAL), a novel extensible collaborative
perception framework. HEAL first establishes a unified feature space with
initial agents via a novel multi-scale foreground-aware Pyramid Fusion network.
When heterogeneous new agents emerge with previously unseen modalities or
models, we align them to the established unified space with an innovative
backward alignment. This step only involves individual training on the new
agent type, thus presenting extremely low training costs and high
extensibility. It also protects new agents' model details from disclosure since
the training can be conducted by the agent owner locally. To enrich agents'
data heterogeneity, we bring OPV2V-H, a new large-scale dataset with more
diverse sensor types. Extensive experiments on OPV2V-H and DAIR-V2X datasets
show that HEAL surpasses SOTA methods in performance while reducing the
training parameters by 91.5% when integrating 3 new agent types. Code and data
are available at: this https URL

如何将不断出现的新异构代理类型纳入协作感知，同时确保高感知性能和低集成成本？本文提出了一种新的可扩展协作感知框架 HEAL，通过建立统一的特征空间和创新的反向对齐来解决该问题。在 OPV2V-H 和 DAIR-V2X 数据集上的实验表明，HEAL 在性能上超越了目前方法，同时将训练参数减少了 91.5％。