Prediction APIs offered for a fee are a fast-growing industry and an
important part of machine learning as a service. While many such services are
available, the heterogeneity in their price and performance makes it
challenging for users to decide which API or combination of APIs to use for
their own data and budget. We take a first step towards addressing this
challenge by proposing FrugalML, a principled framework that jointly learns the
strength and weakness of each API on different data, and performs an efficient
optimization to automatically identify the best sequential strategy to
adaptively use the available APIs within a budget constraint. Our theoretical
analysis shows that natural sparsity in the formulation can be leveraged to
make FrugalML efficient. We conduct systematic experiments using ML APIs from
Google, Microsoft, Amazon, IBM, Baidu and other providers for tasks including
facial emotion recognition, sentiment analysis and speech recognition. Across
various tasks, FrugalML can achieve up to 90% cost reduction while matching the
accuracy of the best single API, or up to 5% better accuracy while matching the
best API's cost.

FrugalML 提出了一种基于机器学习 API 的预测的经济优化框架，能够自动识别可获得的 API 中最优的顺序策略，并在预算约束下实现高准确性和成本效益。

FrugalML：如何更准确便宜地使用 ML 预测 API

FrugalML: How to Use ML Prediction APIs More Accurately and Cheaply

Machine learning (ML) applications are increasingly prevalent. Protecting the
confidentiality of ML models becomes paramount for two reasons: (a) a model can
be a business advantage to its owner, and (b) an adversary may use a stolen
model to find transferable adversarial examples that can evade classification
by the original model. Access to the model can be restricted to be only via
well-defined prediction APIs. Nevertheless, prediction APIs still provide
enough information to allow an adversary to mount model extraction attacks by
sending repeated queries via the prediction API. In this paper, we describe new
model extraction attacks using novel approaches for generating synthetic
queries, and optimizing training hyperparameters. Our attacks outperform
state-of-the-art model extraction in terms of transferability of both targeted
and non-targeted adversarial examples (up to +29-44 percentage points, pp), and
prediction accuracy (up to +46 pp) on two datasets. We provide take-aways on
how to perform effective model extraction attacks. We then propose PRADA, the
first step towards generic and effective detection of DNN model extraction
attacks. It analyzes the distribution of consecutive API queries and raises an
alarm when this distribution deviates from benign behavior. We show that PRADA
can detect all prior model extraction attacks with no false positives.

本文描述了一种新型机器学习模型抽取攻击的方法，并提出了一种名为 PRADA 的检测模型抽取攻击的方法，该方法可以准确检测到之前的模型抽取攻击，且无误报。