Federated Learning has become an important learning paradigm due to its
privacy and computational benefits. As the field advances, two key challenges
that still remain to be addressed are: (1) system heterogeneity - variability
in the compute and/or data resources present on each client, and (2) lack of
labeled data in certain federated settings. Several recent developments have
tried to overcome these challenges independently. In this work, we propose a
unified and systematic framework, \emph{Heterogeneous Self-supervised Federated
Learning} (Hetero-SSFL) for enabling self-supervised learning with federation
on heterogeneous clients. The proposed framework allows collaborative
representation learning across all the clients without imposing architectural
constraints or requiring presence of labeled data. The key idea in Hetero-SSFL
is to let each client train its unique self-supervised model and enable the
joint learning across clients by aligning the lower dimensional representations
on a common dataset. The entire training procedure could be viewed as self and
peer-supervised as both the local training and the alignment procedures do not
require presence of any labeled data. As in conventional self-supervised
learning, the obtained client models are task independent and can be used for
varied end-tasks. We provide a convergence guarantee of the proposed framework
for non-convex objectives in heterogeneous settings and also empirically
demonstrate that our proposed approach outperforms the state of the art methods
by a significant margin.

提出了一种名为 Heterogeneous Self-supervised Federated Learning（Hetero-SSFL）的统一框架，可以在异构客户端上进行协作表示学习，同时解决了系统异质性和标记数据匮乏等问题，并在非凸目标的异构性设置中提供了收敛保证，而且比现有方法表现更佳。

异构客户端的联邦自监督学习

Federated Self-supervised Learning for Heterogeneous Clients

Crowd counting is a fundamental yet challenging task, which desires rich
information to generate pixel-wise crowd density maps. However, most previous
methods only used the limited information of RGB images and cannot well
discover potential pedestrians in unconstrained scenarios. In this work, we
find that incorporating optical and thermal information can greatly help to
recognize pedestrians. To promote future researches in this field, we introduce
a large-scale RGBT Crowd Counting (RGBT-CC) benchmark, which contains 2,030
pairs of RGB-thermal images with 138,389 annotated people. Furthermore, to
facilitate the multimodal crowd counting, we propose a cross-modal
collaborative representation learning framework, which consists of multiple
modality-specific branches, a modality-shared branch, and an Information
Aggregation-Distribution Module (IADM) to capture the complementary information
of different modalities fully. Specifically, our IADM incorporates two
collaborative information transfers to dynamically enhance the modality-shared
and modality-specific representations with a dual information propagation
mechanism. Extensive experiments conducted on the RGBT-CC benchmark demonstrate
the effectiveness of our framework for RGBT crowd counting. Moreover, the
proposed approach is universal for multimodal crowd counting and is also
capable to achieve superior performance on the ShanghaiTechRGBD dataset.
Finally, our source code and benchmark are released at
{http://lingboliu.com/RGBT_Crowd_Counting.html}.

本篇研究通过使用 RGBT 图像和多模态协作表征学习框架来实现人群计数，并提供一个大规模 RGBT 人群计数基准测试数据集，研究表明该框架在 RGBT 人群计数和多模态人群计数方面表现优异。