As Internet of Things (IoT) technology advances, end devices like sensors and
smartphones are progressively equipped with AI models tailored to their local
memory and computational constraints. Local inference reduces communication
costs and latency; however, these smaller models typically underperform
compared to more sophisticated models deployed on edge servers or in the cloud.
Cooperative Inference Systems (CISs) address this performance trade-off by
enabling smaller devices to offload part of their inference tasks to more
capable devices. These systems often deploy hierarchical models that share
numerous parameters, exemplified by Deep Neural Networks (DNNs) that utilize
strategies like early exits or ordered dropout. In such instances, Federated
Learning (FL) may be employed to jointly train the models within a CIS. Yet,
traditional training methods have overlooked the operational dynamics of CISs
during inference, particularly the potential high heterogeneity in serving
rates across clients. To address this gap, we propose a novel FL approach
designed explicitly for use in CISs that accounts for these variations in
serving rates. Our framework not only offers rigorous theoretical guarantees,
but also surpasses state-of-the-art (SOTA) training algorithms for CISs,
especially in scenarios where inference request rates or data availability are
uneven among clients.

通过在合作推理系统中考虑服务速率的差异，我们的新型联邦学习方法针对 IoT 技术下的边缘设备设计，不仅提供严格的理论保证，而且超越了目前合作推理系统的最先进训练算法，尤其是在推理请求率或数据可用性在客户端之间不均匀的场景下。