Web-scale applications can ship code on a daily to weekly cadence. These
applications rely on online metrics to monitor the health of new releases.
Regressions in metric values need to be detected and diagnosed as early as
possible to reduce the disruption to users and product owners. Regressions in
metrics can surface due to a variety of reasons: genuine product regressions,
changes in user population, and bias due to telemetry loss (or processing) are
among the common causes. Diagnosing the cause of these metric regressions is
costly for engineering teams as they need to invest time in finding the root
cause of the issue as soon as possible. We present Lumos, a Python library
built using the principles of AB testing to systematically diagnose metric
regressions to automate such analysis. Lumos has been deployed across the
component teams in Microsoft's Real-Time Communication applications Skype and
Microsoft Teams. It has enabled engineering teams to detect 100s of real
changes in metrics and reject 1000s of false alarms detected by anomaly
detectors. The application of Lumos has resulted in freeing up as much as 95%
of the time allocated to metric-based investigations. In this work, we open
source Lumos and present our results from applying it to two different
components within the RTC group over millions of sessions. This general library
can be coupled with any production system to manage the volume of alerting
efficiently.

使用 AB 测试原理构建的 Python 库 Lumos 可以自动化诊断度量回归，通过在微软的 RTC 应用程序 Skype 和 Microsoft Teams 上实施，Lumos 已经使工程团队在度量基础调查中释放了高达 95% 的时间。

Lumos: 适用于诊断 Web 规模应用中指标回归的库

Lumos: A Library for Diagnosing Metric Regressions in Web-Scale  Applications

The remarkable progress of network embedding has led to state-of-the-art
algorithms in recommendation. However, the sparsity of user-item interactions
(i.e., explicit preferences) on websites remains a big challenge for predicting
users' behaviors. Although research efforts have been made in utilizing some
auxiliary information (e.g., social relations between users) to solve the
problem, the existing rich heterogeneous auxiliary relationships are still not
fully exploited. Moreover, previous works relied on linearly combined
regularizers and suffered parameter tuning.
In this work, we collect abundant relationships from common user behaviors
and item information, and propose a novel framework named IntentGC to leverage
both explicit preferences and heterogeneous relationships by graph
convolutional networks. In addition to the capability of modeling
heterogeneity, IntentGC can learn the importance of different relationships
automatically by the neural model in a nonlinear sense. To apply IntentGC to
web-scale applications, we design a faster graph convolutional model named
IntentNet by avoiding unnecessary feature interactions. Empirical experiments
on two large-scale real-world datasets and online A/B tests in Alibaba
demonstrate the superiority of our method over state-of-the-art algorithms.

本文提出了一种名为 IntentGC 的新型框架，利用图卷积网络来利用用户行为和物品信息之间的明确偏好和异构关系，同时学习模型中不同关系的重要性，将其应用于网络规模应用，设计了一个更快的图卷积模型 IntentNet。实验证明 IntentGC 优越于现有算法。