Machine Learning (ML), particularly deep learning, has seen vast
advancements, leading to the rise of Machine Learning-Enabled Systems (MLS).
However, numerous software engineering challenges persist in propelling these
MLS into production, largely due to various run-time uncertainties that impact
the overall Quality of Service (QoS). These uncertainties emanate from ML
models, software components, and environmental factors. Self-adaptation
techniques present potential in managing run-time uncertainties, but their
application in MLS remains largely unexplored. As a solution, we propose the
concept of a Machine Learning Model Balancer, focusing on managing
uncertainties related to ML models by using multiple models. Subsequently, we
introduce AdaMLS, a novel self-adaptation approach that leverages this concept
and extends the traditional MAPE-K loop for continuous MLS adaptation. AdaMLS
employs lightweight unsupervised learning for dynamic model switching, thereby
ensuring consistent QoS. Through a self-adaptive object detection system
prototype, we demonstrate AdaMLS's effectiveness in balancing system and model
performance. Preliminary results suggest AdaMLS surpasses naive and single
state-of-the-art models in QoS guarantees, heralding the advancement towards
self-adaptive MLS with optimal QoS in dynamic environments.

通过使用多个模型，我们提出了一种用于管理与机器学习模型相关的运行时不确定性的机器学习模型平衡器的概念，并引入了 AdaMLS，一种新颖的自适应方法，它利用这个概念并扩展了传统的 MAPE-K 循环以实现连续的机器学习系统自适应。AdaMLS 通过使用轻量级无监督学习进行动态模型切换，从而确保了一致的服务质量。通过一个自适应目标检测系统原型，我们展示了 AdaMLS 在平衡系统和模型性能方面的有效性。初步结果表明，AdaMLS 在服务质量保证方面超过了天真方法和单一最先进的模型，标志着在动态环境中实现自适应机器学习系统的优化服务质量的进展。