As Internet of Things (IoT) technology advances, end devices like sensors and
smartphones are progressively equipped with AI models tailored to their local
memory and computational constraints. Local inference reduces communication
costs and latency; however, these smaller models typically underperform
compared to more sophisticated models deployed on edge servers or in the cloud.
Cooperative Inference Systems (CISs) address this performance trade-off by
enabling smaller devices to offload part of their inference tasks to more
capable devices. These systems often deploy hierarchical models that share
numerous parameters, exemplified by Deep Neural Networks (DNNs) that utilize
strategies like early exits or ordered dropout. In such instances, Federated
Learning (FL) may be employed to jointly train the models within a CIS. Yet,
traditional training methods have overlooked the operational dynamics of CISs
during inference, particularly the potential high heterogeneity in serving
rates across clients. To address this gap, we propose a novel FL approach
designed explicitly for use in CISs that accounts for these variations in
serving rates. Our framework not only offers rigorous theoretical guarantees,
but also surpasses state-of-the-art (SOTA) training algorithms for CISs,
especially in scenarios where inference request rates or data availability are
uneven among clients.

通过在合作推理系统中考虑服务速率的差异，我们的新型联邦学习方法针对 IoT 技术下的边缘设备设计，不仅提供严格的理论保证，而且超越了目前合作推理系统的最先进训练算法，尤其是在推理请求率或数据可用性在客户端之间不均匀的场景下。

联合学习在合作推理系统中的应用：以早退出网络为例

Federated Learning for Cooperative Inference Systems: The Case of Early  Exit Networks

Amidst the swift advancements in photography and sensor technologies,
high-definition cameras have become commonplace in the deployment of Unmanned
Aerial Vehicles (UAVs) for diverse operational purposes. Within the domain of
UAV imagery analysis, the segmentation of ultra-high resolution images emerges
as a substantial and intricate challenge, especially when grappling with the
constraints imposed by GPU memory-restricted computational devices. This paper
delves into the intricate problem of achieving efficient and effective
segmentation of ultra-high resolution UAV imagery, while operating under
stringent GPU memory limitation. The strategy of existing approaches is to
downscale the images to achieve computationally efficient segmentation.
However, this strategy tends to overlook smaller, thinner, and curvilinear
regions. To address this problem, we propose a GPU memory-efficient and
effective framework for local inference without accessing the context beyond
local patches. In particular, we introduce a novel spatial-guided
high-resolution query module, which predicts pixel-wise segmentation results
with high quality only by querying nearest latent embeddings with the guidance
of high-resolution information. Additionally, we present an efficient
memory-based interaction scheme to correct potential semantic bias of the
underlying high-resolution information by associating cross-image contextual
semantics. For evaluation of our approach, we perform comprehensive experiments
over public benchmarks and achieve superior performance under both conditions
of small and large GPU memory usage limitations. We will release the model and
codes in the future.

该论文阐述了在 GPU 内存限制的情况下，实现高效而有效的超高分辨率无人机图像分割的复杂问题，提出了一种新颖的空间引导式高分辨率查询模块和高效的基于内存的交互方案来改进现有方法中较小、较细和曲线区域的分割效果。