Multi-tenant machine learning services have become emerging data-intensive
workloads in data centers with heavy usage of GPU resources. Due to the large
scale, many tuning parameters and heavy resource usage, it is usually
impractical to evaluate and benchmark those machine learning services on real
clusters. In this demonstration, we present AnalySIM, a cluster simulator that
allows efficient design explorations for multi-tenant machine learning
services. Specifically, by trace-driven cluster workload simulation, AnalySIM
can easily test and analyze various scheduling policies in a number of
performance metrics such as GPU resource utilization. AnalySIM simulates the
cluster computational resource based on both physical topology and logical
partition. The tool has been used in SenseTime to understand the impact of
different scheduling policies with the trace from a real production cluster of
over 1000 GPUs. We find that preemption and migration are able to significantly
reduce average job completion time and mitigate the resource fragmentation
problem.

AnalySIM 是一个集群模拟器，可通过跟踪驱动的集群工作负载模拟，以各种性能指标（如 GPU 资源利用率）测试和分析各种调度策略，具有物理拓扑和逻辑分区两种基于集群计算资源的模拟方式。

一个面向千个 GPU 的多租户机器学习服务模拟平台

A Simulation Platform for Multi-tenant Machine Learning Services on Thousands of GPUs

Cloud vendors are increasingly offering machine learning services as part of
their platform and services portfolios. These services enable the deployment of
machine learning models on the cloud that are offered on a pay-per-query basis
to application developers and end users. However recent work has shown that the
hosted models are susceptible to extraction attacks. Adversaries may launch
queries to steal the model and compromise future query payments or privacy of
the training data. In this work, we present a cloud-based extraction monitor
that can quantify the extraction status of models by observing the query and
response streams of both individual and colluding adversarial users. We present
a novel technique that uses information gain to measure the model learning rate
by users with increasing number of queries. Additionally, we present an
alternate technique that maintains intelligent query summaries to measure the
learning rate relative to the coverage of the input feature space in the
presence of collusion. Both these approaches have low computational overhead
and can easily be offered as services to model owners to warn them of possible
extraction attacks from adversaries. We present performance results for these
approaches for decision tree models deployed on BigML MLaaS platform, using
open source datasets and different adversarial attack strategies.

该研究提出了一种基于云的提取监视器，通过观察单个和串通的对手用户的查询和响应流来量化模型的提取状态，从而使用信息增益来测量具有不断增加查询数量的用户的模型学习速率，并维护智能查询摘要以在串通存在的情况下测量与输入特征空间覆盖度相关的学习速率，以提醒模型所有者可能存在侵犯攻击。