Restless multi-arm bandits (RMABs) is a popular decision-theoretic framework
that has been used to model real-world sequential decision making problems in
public health, wildlife conservation, communication systems, and beyond.
Deployed RMAB systems typically operate in two stages: the first predicts the
unknown parameters defining the RMAB instance, and the second employs an
optimization algorithm to solve the constructed RMAB instance.
In this work we provide and analyze the results from a first-of-its-kind
deployment of an RMAB system in public health domain, aimed at improving
maternal and child health. Our analysis is focused towards understanding the
relationship between prediction accuracy and overall performance of deployed
RMAB systems. This is crucial for determining the value of investing in
improving predictive accuracy towards improving the final system performance,
and is useful for diagnosing, monitoring deployed RMAB systems.
Using real-world data from our deployed RMAB system, we demonstrate that an
improvement in overall prediction accuracy may even be accompanied by a
degradation in the performance of RMAB system -- a broad investment of
resources to improve overall prediction accuracy may not yield expected
results. Following this, we develop decision-focused evaluation metrics to
evaluate the predictive component and show that it is better at explaining
(both empirically and theoretically) the overall performance of a deployed RMAB
system.

研究 RMAB 系统在公共卫生领域的部署，发现整体预测准确性的提高可能会伴随着 RMAB 系统性能的降低；使用决策评估度量来评估预测组件，更好地解释了 RMAB 系统的整体性能表现。