Neural networks excel at discovering statistical patterns in high-dimensional
data sets. In practice, higher-order cumulants, which quantify the non-Gaussian
correlations between three or more variables, are particularly important for
the performance of neural networks. But how efficient are neural networks at
extracting features from higher-order cumulants? We study this question in the
spiked cumulant model, where the statistician needs to recover a privileged
direction or "spike" from the order-$p\ge 4$ cumulants of~$d$-dimensional
inputs. We first characterise the fundamental statistical and computational
limits of recovering the spike by analysing the number of samples~$n$ required
to strongly distinguish between inputs from the spiked cumulant model and
isotropic Gaussian inputs. We find that statistical distinguishability requires
$n\gtrsim d$ samples, while distinguishing the two distributions in polynomial
time requires $n \gtrsim d^2$ samples for a wide class of algorithms, i.e.
those covered by the low-degree conjecture. These results suggest the existence
of a wide statistical-to-computational gap in this problem. Numerical
experiments show that neural networks learn to distinguish the two
distributions with quadratic sample complexity, while "lazy" methods like
random features are not better than random guessing in this regime. Our results
show that neural networks extract information from higher-order correlations in
the spiked cumulant model efficiently, and reveal a large gap in the amount of
data required by neural networks and random features to learn from higher-order
cumulants.

神经网络在提取高阶累积量中的信息效率很高，并揭示了神经网络和随机特征在学习高阶累积量所需数据量方面的巨大差距。

高阶统计的高效学习：假设检验、随机特征和神经网络

Learning from higher-order statistics, efficiently: hypothesis tests,  random features, and neural networks

Causal discovery with latent variables is a crucial but challenging task.
Despite the emergence of numerous methods aimed at addressing this challenge,
they are not fully identified to the structure that two observed variables are
influenced by one latent variable and there might be a directed edge in
between. Interestingly, we notice that this structure can be identified through
the utilization of higher-order cumulants. By leveraging the higher-order
cumulants of non-Gaussian data, we provide an analytical solution for
estimating the causal coefficients or their ratios. With the estimated (ratios
of) causal coefficients, we propose a novel approach to identify the existence
of a causal edge between two observed variables subject to latent variable
influence. In case when such a causal edge exits, we introduce an asymmetry
criterion to determine the causal direction. The experimental results
demonstrate the effectiveness of our proposed method.

通过利用非高斯数据的高阶累积量，我们提出一种用于估计因果系数或其比例的解析解，并通过估计后的（比例）因果系数，提出一种新的方法来确定受潜变量影响的两个观测变量之间是否存在因果边，以及在存在这样的因果边时引入了一种非对称性准则来确定因果方向，实验结果表明了我们提出方法的有效性。

基于高阶累积量的潜变量因果结构识别

Identification of Causal Structure with Latent Variables Based on Higher  Order Cumulants

Causal discovery with latent confounders is an important but challenging task
in many scientific areas. Despite the success of some overcomplete independent
component analysis (OICA) based methods in certain domains, they are
computationally expensive and can easily get stuck into local optima. We notice
that interestingly, by making use of higher-order cumulants, there exists a
closed-form solution to OICA in specific cases, e.g., when the mixing procedure
follows the One-Latent-Component structure. In light of the power of the
closed-form solution to OICA corresponding to the One-Latent-Component
structure, we formulate a way to estimate the mixing matrix using the
higher-order cumulants, and further propose the testable One-Latent-Component
condition to identify the latent variables and determine causal orders. By
iteratively removing the share identified latent components, we successfully
extend the results on the One-Latent-Component structure to the
Multi-Latent-Component structure and finally provide a practical and
asymptotically correct algorithm to learn the causal structure with latent
variables. Experimental results illustrate the asymptotic correctness and
effectiveness of the proposed method.

针对具有潜在混杂因素的因果关系发现问题，通过利用高阶累积量，提出一种基于独立分量分析的解决方法，可以自动检测潜在变量，确定因果顺序，并扩展到多个潜在变量的情况下。实验明确了该方法的正确性和有效性。