Automatically evaluating the quality of responses in open-domain dialogue
systems is a challenging but crucial task. Current evaluation metrics often
fail to align with human judgments, especially when assessing responses that
are grammatically correct. To address this issue, we propose a novel metric,
called CausalScore, which assesses the relevance of responses by measuring the
causal strength between dialogue histories and responses. The causal strength
is estimated by utilizing both unconditional dependence and conditional
dependencies from the dialogue history to responses. We compare our metric with
the existing competitive metrics in terms of their alignment with human
judgements. Our experimental results demonstrate that CausalScore significantly
surpasses existing state-of-the-art metrics by aligning better with human
judgements. Additionally, we collect a new dialogue dataset CGDIALOG+ with
human-annotated causal relations and a set of pairwise human judgements to
facilitate the development of future automatic metrics.

自动评估开放领域对话系统中回应的质量是一项具有挑战性但又至关重要的任务。我们提出了一种新颖的度量标准，称为 CausalScore，通过测量对话历史和回应之间的因果关系强度来评估回应的相关性。我们的实验结果表明，CausalScore 在与人类判断的一致性方面显著优于现有的领先指标，此外，我们还收集了一个带有人工注释的因果关系对话数据集 CGDIALOG + 以及一组成对人类判断，以促进未来自动度量标准的发展。

CausalScore: 用于评估开放领域对话系统中回复相关性的自动无参考度量

CausalScore: An Automatic Reference-Free Metric for Assessing Response  Relevance in Open-Domain Dialogue Systems

We introduce Dagma-DCE, an interpretable and model-agnostic scheme for
differentiable causal discovery. Current non- or over-parametric methods in
differentiable causal discovery use opaque proxies of ``independence'' to
justify the inclusion or exclusion of a causal relationship. We show
theoretically and empirically that these proxies may be arbitrarily different
than the actual causal strength. Juxtaposed to existing differentiable causal
discovery algorithms, \textsc{Dagma-DCE} uses an interpretable measure of
causal strength to define weighted adjacency matrices. In a number of simulated
datasets, we show our method achieves state-of-the-art level performance. We
additionally show that \textsc{Dagma-DCE} allows for principled thresholding
and sparsity penalties by domain-experts. The code for our method is available
open-source at this https URL, and can easily be
adapted to arbitrary differentiable models.

我们介绍了一种可解释且模型无关的梯度可逆因果发现方法 Dagma-DCE，它使用可解释的因果强度来定义加权邻接矩阵，并在多个模拟数据集上展现了与其他方法相媲美的性能水平。

Dagma-DCE：可解释、非参数可微因果发现

Dagma-DCE: Interpretable, Non-Parametric Differentiable Causal Discovery

The recent criticisms of the robustness of post hoc model approximation
explanation methods (like LIME and SHAP) have led to the rise of model-precise
abductive explanations. For each data point, abductive explanations provide a
minimal subset of features that are sufficient to generate the outcome. While
theoretically sound and rigorous, abductive explanations suffer from a major
issue -- there can be several valid abductive explanations for the same data
point. In such cases, providing a single abductive explanation can be
insufficient; on the other hand, providing all valid abductive explanations can
be incomprehensible due to their size. In this work, we solve this issue by
aggregating the many possible abductive explanations into feature importance
scores. We propose three aggregation methods: two based on power indices from
cooperative game theory and a third based on a well-known measure of causal
strength. We characterize these three methods axiomatically, showing that each
of them uniquely satisfies a set of desirable properties. We also evaluate them
on multiple datasets and show that these explanations are robust to the attacks
that fool SHAP and LIME.

通过合作博弈理论和因果强度的措施，将多个可能的诱导解释汇总为特征重要性评分，从而解决了单个和全部有效诱导解释之间的矛盾，这些解释对多个数据集都具有鲁棒性，同时欺骗 SHAP 和 LIME 的攻击也是有效的。

归纳解释的公理化聚合

Axiomatic Aggregations of Abductive Explanations

Many methods for causal inference generate directed acyclic graphs (DAGs)
that formalize causal relations between $n$ variables. Given the joint
distribution on all these variables, the DAG contains all information about how
intervening on one variable changes the distribution of the other $n-1$
variables. However, quantifying the causal influence of one variable on another
one remains a nontrivial question. Here we propose a set of natural, intuitive
postulates that a measure of causal strength should satisfy. We then introduce
a communication scenario, where edges in a DAG play the role of channels that
can be locally corrupted by interventions. Causal strength is then the relative
entropy distance between the old and the new distribution. Many other measures
of causal strength have been proposed, including average causal effect,
transfer entropy, directed information, and information flow. We explain how
they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally,
we investigate the behavior of our measure on time-series, supporting our
claims with experiments on simulated data.

本文提出了一组自然、直观的因果强度度量法，并通过时间序列的模拟实验证实了我们提出的度量法的有效性。