Although the computing power of mobile devices is increasing, machine
learning models are also growing in size. This trend creates problems for
mobile devices due to limitations like their memory capacity and battery life.
While many services, like ChatGPT and Midjourney, run all the inferences in the
cloud, we believe a flexible and fine-grained task distribution is more
desirable. In this work, we consider model segmentation as a solution to
improving the user experience, dividing the computation between mobile devices
and the cloud in a way that offloads the compute-heavy portion of the model
while minimizing the data transfer required. We show that the division not only
reduces the wait time for users but can also be fine-tuned to optimize the
workloads of the cloud. To achieve that, we design a scheduler that collects
information about network quality, client device capability, and job
requirements, making decisions to achieve consistent performance across a range
of devices while reducing the work the cloud needs to perform.

通过模型分割和设计调度程序，我们将计算任务在移动设备和云端之间进行灵活而细粒度的分配，从而既减少用户的等待时间，又优化云端的工作负载。

云移结合 在机器学习中的应用

Combining Cloud and Mobile Computing for Machine Learning

The edge intelligence (EI) has been widely applied recently. Spliting the
model between device, edge server, and cloud can improve the performance of EI
greatly. The model segmentation without user mobility has been investigated
deeply by previous works. However, in most use cases of EI, the end devices are
mobile. Only a few works have been carried out on this aspect. These works
still have many issues, such as ignoring the energy consumption of mobile
device, inappropriate network assumption, and low effectiveness on adaptiving
user mobility, etc. Therefore, for addressing the disadvantages of model
segmentation and resource allocation in previous works, we propose mobility and
cost aware model segmentation and resource allocation algorithm for
accelerating the inference at edge (MCSA). Specfically, in the scenario without
user mobility, the loop interation gradient descent (Li-GD) algorithm is
provided. When the mobile user has a large model inference task needs to be
calculated, it will take the energy consumption of mobile user, the
communication and computing resource renting cost, and the inference delay into
account to find the optimal model segmentation and resource allocation
strategy. In the scenario with user mobility, the mobiity aware Li-GD (MLi-GD)
algorithm is proposed to calculate the optimal strategy. Then, the properties
of the proposed algorithms are investigated, including convergence, complexity,
and approximation ratio. The experimental results demonstrate the effectiveness
of the proposed algorithms.

针对边缘智能化中的模型切割和资源分配存在的问题，我们提出了一种移动和成本感知的模型切割和资源分配算法，用于加速边缘推理。具体来说，在无用户移动的场景中，提供了循环迭代梯度下降（Li-GD）算法；而在存在用户移动的场景中，提出了移动感知的 Li-GD（MLi-GD）算法，以计算最优策略。实验结果证明了提出算法的有效性。