With the rapid development of storage and computing power on mobile devices,
it becomes critical and popular to deploy models on devices to save onerous
communication latencies and to capture real-time features. While quite a lot of
works have explored to facilitate on-device learning and inference, most of
them focus on dealing with response delay or privacy protection. Little has
been done to model the collaboration between the device and the cloud modeling
and benefit both sides jointly. To bridge this gap, we are among the first
attempts to study the Device-Cloud Collaborative Learning (DCCL) framework.
Specifically, we propose a novel MetaPatch learning approach on the device side
to efficiently achieve "thousands of people with thousands of models" given a
centralized cloud model. Then, with billions of updated personalized device
models, we propose a "model-over-models" distillation algorithm, namely
MoMoDistill, to update the centralized cloud model. Our extensive experiments
over a range of datasets with different settings demonstrate the effectiveness
of such collaboration on both cloud and devices, especially its superiority to
model long-tailed users.

本文提出了一种名为 “Device-Cloud Collaborative Learning” 的框架，运用 “MetaPatch 学习” 及 “MoMoDistill” 算法实现了设备端和云端之间的合作学习，取得了较好的效果。