As AI and machine-learned software are used increasingly for making decisions that affect humans, it is imperative that they remain fair and unbiased in their decisions. To complement design-time bias mitigation measures, runtime verification techniques have been introduced recently to monitor the algorithmic fairness of deployed systems. Previous monitoring techniques assume full observability of the states of the (unknown) monitored system. Moreover, they can monitor only fairness properties that are specified as arithmetic expressions over the probabilities of different events. In this work, we extend fairness monitoring to systems modeled as partially observed Markov chains (POMC), and to specifications containing arithmetic expressions over the expected values of numerical functions on event sequences. The only assumptions we make are that the underlying POMC is aperiodic and starts in the stationary distribution, with a bound on its mixing time being known. These assumptions enable us to estimate a given property for the entire distribution of possible executions of the monitored POMC, by observing only a single execution. Our monitors observe a long run of the system and, after each new observation, output updated PAC-estimates of how fair or biased the system is. The monitors are computationally lightweight and, using a prototype implementation, we demonstrate their effectiveness on several real-world examples.

随着人工智能和机器学习软件在影响人类的决策中的应用日益增多，保持公正无偏成为当务之急。为了补充设计时的偏见缓解措施，最近引入了运行时验证技术来监测部署系统的算法公平性。本研究将公平性监测扩展到部分观测的马尔可夫链(POMC)建模的系统，并且适用于包含事件序列中数值函数期望值的算术表达式的规范。我们的监测器通过仅观测到单个执行，估计了受监测的POMC可能执行的整个分布的给定属性。这些监测器在每次新观测后输出更新的PAC估计，以评估系统的公平性或偏见。监测器计算量轻，并使用原型实现在几个真实世界的示例中展示了其有效性。

在部分观察下监控算法公平性