In this paper we demonstrate the possibility of a trend reversal in binary
classification tasks between the dataset and a classification score obtained
from a trained model. This trend reversal occurs for certain choices of the
regularization parameter for model training, namely, if the parameter is
contained in what we call the pathological regularization regime. For ridge
regression, we give necessary and sufficient algebraic conditions on the
dataset for the existence of a pathological regularization regime. Moreover,
our results provide a data science practitioner with a hands-on tool to avoid
hyperparameter choices suffering from trend reversal. We furthermore present
numerical results on pathological regularization regimes for logistic
regression. Finally, we draw connections to datasets exhibiting Simpson's
paradox, providing a natural source of pathological datasets.

该研究论文通过展示训练模型获得的分类分数与数据集之间的趋势逆转的可能性，阐述了模型训练中正则化参数的选择对趋势逆转发生的影响，提供数据科学从业者避免遭受趋势逆转的超参数选择的实用工具，并通过逻辑回归在病态正则化区间上的数值结果，从路径性数据集的角度探讨了辛普森悖论的关联。

分类任务中的病理化正则化方案

Pathological Regularization Regimes in Classification Tasks

Simpson's paradox is an obstacle to establishing a probabilistic association
between two events $a_1$ and $a_2$, given the third (lurking) random variable
$B$. We focus on scenarios when the random variables $A$ (which combines $a_1$,
$a_2$, and their complements) and $B$ have a common cause $C$ that need not be
observed. Alternatively, we can assume that $C$ screens out $A$ from $B$. For
such cases, the correct association between $a_1$ and $a_2$ is to be defined
via conditioning over $C$. This set-up generalizes the original Simpson's
paradox. Now its two contradicting options simply refer to two particular and
different causes $C$. We show that if $B$ and $C$ are binary and $A$ is
quaternary (the minimal and the most widespread situation for valid Simpson's
paradox), the conditioning over any binary common cause $C$ establishes the
same direction of the association between $a_1$ and $a_2$ as the conditioning
over $B$ in the original formulation of the paradox. Thus, for the minimal
common cause, one should choose the option of Simpson's paradox that assumes
conditioning over $B$ and not its marginalization. For tertiary (unobserved)
common causes $C$ all three options of Simpson's paradox become possible (i.e.
marginalized, conditional, and none of them), and one needs prior information
on $C$ to choose the right option.

辛普森悖论中，随机变量与共因变量的关系建立在共同原因的条件下，而非辖权化。对于一个最小的共同原因，应选择辛普森悖论的条件分布选项，而不是辖权化。对于三元（未观察到）的共同原因，辛普森悖论的三个选项都可能存在，需根据共同原因的先验信息选择正确的选项。

通过共因原则解决辛普森悖论

Resolution of Simpson's paradox via the common cause principle

When we compare the influences of two causes on an outcome, if the conclusion
from every group is against that from the conflation, we think there is
Simpson's Paradox. The Existing Causal Inference Theory (ECIT) can make the
overall conclusion consistent with the grouping conclusion by removing the
confounder's influence to eliminate the paradox. The ECIT uses relative risk
difference Pd = max(0, (R - 1)/R) (R denotes the risk ratio) as the probability
of causation. In contrast, Philosopher Fitelson uses confirmation measure D
(posterior probability minus prior probability) to measure the strength of
causation. Fitelson concludes that from the perspective of Bayesian
confirmation, we should directly accept the overall conclusion without
considering the paradox. The author proposed a Bayesian confirmation measure b*
similar to Pd before. To overcome the contradiction between the ECIT and
Bayesian confirmation, the author uses the semantic information method with the
minimum cross-entropy criterion to deduce causal confirmation measure Cc = (R
-1)/max(R, 1). Cc is like Pd but has normalizing property (between -1 and 1)
and cause symmetry. It especially fits cases where a cause restrains an
outcome, such as the COVID-19 vaccine controlling the infection. Some examples
(about kidney stone treatments and COVID-19) reveal that Pd and Cc are more
reasonable than D; Cc is more useful than Pd.

本文主要介绍了关于 Simpson's Paradox 与 Causal Inference Theory 的理论，利用 Bayesian confirmation 方法来推导出一种新的因果证实度量 Cc，与 Pd 比较更为合理和实用，特别适用于控制性因果关系，如 COVID-19 疫苗控制感染和肾结石治疗等方面。