Research is facing a reproducibility crisis, in which the results and
findings of many studies are difficult or even impossible to reproduce. This is
also the case in machine learning (ML) and artificial intelligence (AI)
research. Often, this is the case due to unpublished data and/or source-code,
and due to sensitivity to ML training conditions. Although different solutions
to address this issue are discussed in the research community such as using ML
platforms, the level of reproducibility in ML-driven research is not increasing
substantially. Therefore, in this mini survey, we review the literature on
reproducibility in ML-driven research with three main aims: (i) reflect on the
current situation of ML reproducibility in various research fields, (ii)
identify reproducibility issues and barriers that exist in these research
fields applying ML, and (iii) identify potential drivers such as tools,
practices, and interventions that support ML reproducibility. With this, we
hope to contribute to decisions on the viability of different solutions for
supporting ML reproducibility.

面对研究的可重复性危机，机器学习和人工智能研究也面临同样的问题。虽然研究社区已经提出了不同的解决方案如使用机器学习平台，但机器学习驱动的研究的可重复性水平并没有显著提高。本文通过综述相关文献，讨论机器学习驱动研究中的可重复性问题和障碍，并探索工具、实践和干预等潜在驱动因素，提供对于支持机器学习可重复性的不同解决方案可行性的决策支持。

机器学习驱动研究的可重现性

Reproducibility in Machine Learning-Driven Research

ML platforms help enable intelligent data-driven applications and maintain
them with limited engineering effort. Upon sufficiently broad adoption, such
platforms reach economies of scale that bring greater component reuse while
improving efficiency of system development and maintenance. For an end-to-end
ML platform with broad adoption, scaling relies on pervasive ML automation and
system integration to reach the quality we term self-serve that we define with
ten requirements and six optional capabilities. With this in mind, we identify
long-term goals for platform development, discuss related tradeoffs and future
work. Our reasoning is illustrated on two commercially-deployed end-to-end ML
platforms that host hundreds of real-time use cases -- one general-purpose and
one specialized.

研究探讨了机器学习平台的发展、可伸缩性以及实现自助服务质量的要求和能力，并结合两个商业化的端到端机器学习平台来说明其推广应用及未来发展的目标。

可扩展的端到端机器学习平台：从自动机器学习到自助式服务

Scalable End-to-End ML Platforms: from AutoML to Self-serve

Data-centric AI is a new and exciting research topic in the AI community, but
many organizations already build and maintain various "data-centric"
applications whose goal is to produce high quality data. These range from
traditional business data processing applications (e.g., "how much should we
charge each of our customers this month?") to production ML systems such as
recommendation engines. The fields of data and ML engineering have arisen in
recent years to manage these applications, and both include many interesting
novel tools and processes. In this paper, we discuss several lessons from data
and ML engineering that could be interesting to apply in data-centric AI, based
on our experience building data and ML platforms that serve thousands of
applications at a range of organizations.

讨论数据与机器学习工程的重要性以及如何应用于数据中心人工智能领域。

数据中心的 AI 能从数据和 ML 工程中学到什么？

What can Data-Centric AI Learn from Data and ML Engineering?

We describe TensorFlow-Serving, a system to serve machine learning models
inside Google which is also available in the cloud and via open-source. It is
extremely flexible in terms of the types of ML platforms it supports, and ways
to integrate with systems that convey new models and updated versions from
training to serving. At the same time, the core code paths around model lookup
and inference have been carefully optimized to avoid performance pitfalls
observed in naive implementations. Google uses it in many production
deployments, including a multi-tenant model hosting service called TFS^2.

TensorFlow-Serving 是一个可用于 Google 内部和云端的机器学习模型服务系统，支持多种 ML 平台，优化模型查找和推理性能，用于多租户模型托管服务 TFS^2 等生产环境。