Serverless computing offers attractive scalability, elasticity and
cost-effectiveness. However, constraints on memory, CPU and function runtime
have hindered its adoption for data-intensive applications and machine learning
(ML) workloads. Traditional 'server-ful' platforms enable distributed
computation via fast networks and well-established inter-process communication
(IPC) mechanisms such as MPI and shared memory. In the absence of such
solutions in the serverless domain, parallel computation with significant IPC
requirements is challenging. We present FSD-Inference, the first fully
serverless and highly scalable system for distributed ML inference. We explore
potential communication channels, in conjunction with Function-as-a-Service
(FaaS) compute, to design a state-of-the-art solution for distributed ML within
the context of serverless data-intensive computing. We introduce novel fully
serverless communication schemes for ML inference workloads, leveraging both
cloud-based publish-subscribe/queueing and object storage offerings. We
demonstrate how publish-subscribe/queueing services can be adapted for FaaS IPC
with comparable performance to object storage, while offering significantly
reduced cost at high parallelism levels. We conduct in-depth experiments on
benchmark DNNs of various sizes. The results show that when compared to
server-based alternatives, FSD-Inference is significantly more cost-effective
and scalable, and can even achieve competitive performance against optimized
HPC solutions. Experiments also confirm that our serverless solution can handle
large distributed workloads and leverage high degrees of FaaS parallelism.

FSD-Inference 是第一个完全无服务器且高度可扩展的分布式机器学习推断系统，通过与 FaaS 计算相结合，探索了潜在的通信渠道，为无服务器数据密集型计算中的分布式机器学习设计了一种先进的解决方案。