Regularization improves generalization of supervised models to out-of-sample data. Prior works have shown that prediction in the causal direction (effect from cause) results in lower testing error than the anti-causal direction. However, existing regularization methods are agnostic of causality. We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE learns the causal directed acyclical graph (DAG) as an adjacency matrix embedded in the neural network's input layers, thereby facilitating the discovery of optimal predictors. Furthermore, CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features. We provide a theoretical generalization bound for our approach and conduct experiments on a plethora of synthetic and real publicly available datasets demonstrating that CASTLE consistently leads to better out-of-sample predictions as compared to other popular benchmark regularizers.

本文提出Causal Structure Learning (CASTLE) regularization对神经网络进行因果关系约束，实现变量之间的因果关系的发现，提高模型泛化能力。CASTLE regularization能有效地恢复只有因果关系的因素，而网格化的正则化方法只能恢复所有输入功能。理论分析和实验证明，与其他流行的基准约束相比，CASTLE能够更好地预测测试集并具有更好的泛化性能。

通过辅助因果图发现实现正则化：CASTLE