Representing uncertainty in causal discovery is a crucial component for experimental design, and more broadly, for safe and reliable causal decision making. Bayesian Causal Discovery (BCD) offers a principled approach to encapsulating this uncertainty. Unlike non-Bayesian causal discovery, which relies on a single estimated causal graph and model parameters for assessment, evaluating BCD presents challenges due to the nature of its inferred quantity - the posterior distribution. As a result, the research community has proposed various metrics to assess the quality of the approximate posterior. However, there is, to date, no consensus on the most suitable metric(s) for evaluation. In this work, we reexamine this question by dissecting various metrics and understanding their limitations. Through extensive empirical evaluation, we find that many existing metrics fail to exhibit a strong correlation with the quality of approximation to the true posterior, especially in scenarios with low sample sizes where BCD is most desirable. We highlight the suitability (or lack thereof) of these metrics under two distinct factors: the identifiability of the underlying causal model and the quantity of available data. Both factors affect the entropy of the true posterior, indicating that the current metrics are less fitting in settings of higher entropy. Our findings underline the importance of a more nuanced evaluation of new methods by taking into account the nature of the true posterior, as well as guide and motivate the development of new evaluation procedures for this challenge.

在因果关系发现中，表示不确定性是实验设计的关键组成部分，而贝叶斯因果发现（BCD）提供了一种原则性的方法来包括这种不确定性。本研究通过剖析各种度量标准，并了解其局限性，通过广泛的实证评估，发现许多现有度量标准在与真实后验分布的逼近质量之间缺乏强关联，尤其是在BCD最受欢迎的低样本量场景中。我们的研究结果强调了根据真实后验分布的性质进行更加细致的新方法评估的重要性，同时指导和推动为这一挑战开发新的评估程序。

贝叶斯因果探索评估中的挑战与考虑