Given two views of data, we consider the problem of finding the features of
one view which can be most faithfully inferred from the other. We find that
these are also the most correlated variables in the sense of deep canonical
correlation analysis (DCCA). Moreover, we show that these variables can be used
to construct a non-parametric representation of the implied joint probability
distribution, which can be thought of as a classical version of the Schmidt
decomposition of quantum states. This representation can be used to compute the
expectations of functions over one view of data conditioned on the other, such
as Bayesian estimators and their standard deviations. We test the approach
using inference on occluded MNIST images, and show that our representation
contains multiple modes. Surprisingly, when applied to supervised learning (one
dataset consists of labels), this approach automatically provides
regularization and faster convergence compared to the cross-entropy objective.
We also explore using this approach to discover salient independent variables
of a single dataset.

通过深度典型相关分析 (DCCA) 找出可以最真实地从一种数据视角推导出的特征，进而构建出一非参数隐含联合概率分布的经典版本，以求解贝叶斯估算和标准偏差，并测试其在 OCCLUD MNIST 图像推理和监督学习中的可行性和效率，同时探索了发现单个数据集中显著的独立变量的可能性。

学习统计推断相关特征

Learning relevant features for statistical inference

This paper derives two new optimization-driven Monte Carlo algorithms
inspired from variable splitting and data augmentation. In particular, the
formulation of one of the proposed approaches is closely related to the
alternating direction method of multipliers (ADMM) main steps. The proposed
framework enables to derive faster and more efficient sampling schemes than the
current state-of-the-art methods and can embed the latter. By sampling
efficiently the parameter to infer as well as the hyperparameters of the
problem, the generated samples can be used to approximate Bayesian estimators
of the parameters to infer. Additionally, the proposed approach brings
confidence intervals at a low cost contrary to optimization methods.
Simulations on two often-studied signal processing problems illustrate the
performance of the two proposed samplers. All results are compared to those
obtained by recent state-of-the-art optimization and MCMC algorithms used to
solve these problems.

本研究提出了两种新的基于优化驱动的蒙特卡罗算法，通过变量分割和数据增强来获得更快、更高效的采样方案，并且生成的样本可用于近似贝叶斯估计。此外，所提出的方法可以用较低成本的方式提供置信区间。两个经常研究的信号处理问题的模拟结果显示了这两种采样器的性能，这些结果与使用最新的优化和 MCMC 算法解决这些问题所获得的结果进行了比较。

分割和增广的 Gibbs 采样器 - 应用于大规模推理问题

Split-and-augmented Gibbs sampler - Application to large-scale inference  problems

We present a novel notion of complexity that interpolates between and
generalizes some classic existing complexity notions in learning theory: for
estimators like empirical risk minimization (ERM) with arbitrary bounded
losses, it is upper bounded in terms of data-independent Rademacher complexity;
for generalized Bayesian estimators, it is upper bounded by the data-dependent
information complexity (also known as stochastic or PAC-Bayesian,
$\mathrm{KL}(\text{posterior} \operatorname{\|} \text{prior})$ complexity. For
(penalized) ERM, the new complexity reduces to (generalized) normalized maximum
likelihood (NML) complexity, i.e. a minimax log-loss individual-sequence
regret. Our first main result bounds excess risk in terms of the new
complexity. Our second main result links the new complexity via Rademacher
complexity to $L_2(P)$ entropy, thereby generalizing earlier results of Opper,
Haussler, Lugosi, and Cesa-Bianchi who did the log-loss case with $L_\infty$.
Together, these results recover optimal bounds for VC- and large (polynomial
entropy) classes, replacing localized Rademacher complexity by a simpler
analysis which almost completely separates the two aspects that determine the
achievable rates: 'easiness' (Bernstein) conditions and model complexity.

提出了一种新的学习理论复杂性概念，它在经验风险最小化和贝叶斯估计器的情况下分别以数据无关的 Rademacher 复杂度和数据相关的信息复杂度进行上限绑定，并通过 Rademacher 复杂度将其与 $L_2 (P)$ 熵进行关联。该研究进一步使用 ' 易于理解 ' 和模型复杂性等相互分离的方法，提供适用于 VC 和多项式熵类的最优性差距上限。