The number of artificial intelligence algorithms for learning causal models
from data is growing rapidly. Most ``causal discovery'' or ``causal structure
learning'' algorithms are primarily validated through simulation studies.
However, no widely accepted simulation standards exist and publications often
report conflicting performance statistics -- even when only considering
publications that simulate data from linear models. In response, several
manuscripts have criticized a popular simulation design for validating
algorithms in the linear case.
We propose a new simulation design for generating linear models for directed
acyclic graphs (DAGs): the DAG-adaptation of the Onion (DaO) method. DaO
simulations are fundamentally different from existing simulations because they
prioritize the distribution of correlation matrices rather than the
distribution of linear effects. Specifically, the DaO method uniformly samples
the space of all correlation matrices consistent with (i.e. Markov to) a DAG.
We also discuss how to sample DAGs and present methods for generating DAGs with
scale-free in-degree or out-degree. We compare the DaO method against two
alternative simulation designs and provide implementations of the DaO method in
Python and R: this https URL We advocate for others
to adopt DaO simulations as a fair universal benchmark.

人工智能算法、因果发现、模拟设计、线性模型、有向无环图（DAG）

用 DAG-Onion 方法改进因果发现的模拟验证

Better Simulations for Validating Causal Discovery with the  DAG-Adaptation of the Onion Method

Simulation studies play a key role in the validation of causal inference
methods. The simulation results are reliable only if the study is designed
according to the promised operational conditions of the method-in-test. Still,
many causal inference literature tend to design over-restricted or misspecified
studies. In this paper, we elaborate on the problem of improper simulation
design for causal methods and compile a list of desiderata for an effective
simulation framework. We then introduce partially-randomized causal simulation
(PARCS), a simulation framework that meets those desiderata. PARCS synthesizes
data based on graphical causal models and a wide range of adjustable
parameters. There is a legible mapping from usual causal assumptions to the
parameters, thus, users can identify and specify the subset of related
parameters and randomize the remaining ones to generate a range of complying
data-generating processes for their causal method. The result is a more
comprehensive and inclusive empirical investigation for causal claims. Using
PARCS, we reproduce and extend the simulation studies of two well-known causal
discovery and missing data analysis papers to emphasize the necessity of a
proper simulation design. Our results show that those papers would have
improved and extended the findings, had they used PARCS for simulation. The
framework is implemented as a Python package, too. By discussing the
comprehensiveness and transparency of PARCS, we encourage causal inference
researchers to utilize it as a standard tool for future works.

依据确定方法的操作条件，使用适当设计的 PARCS 模拟框架，合成基于因果图模型和各种可调参数的数据，进而更全面地检验因果推断。