Many state-of-the-art results obtained with deep networks are achieved with
the largest models that could be trained, and if more computation power was
available, we might be able to exploit much larger datasets in order to improve
generalization ability. Whereas in learning algorithms such as decision trees
the ratio of capacity (e.g., the number of parameters) to computation is very
favorable (up to exponentially more parameters than computation), the ratio is
essentially 1 for deep neural networks. Conditional computation has been
proposed as a way to increase the capacity of a deep neural network without
increasing the amount of computation required, by activating some parameters
and computation "on-demand", on a per-example basis. In this note, we propose a
novel parametrization of weight matrices in neural networks which has the
potential to increase up to exponentially the ratio of the number of parameters
to computation. The proposed approach is based on turning on some parameters
(weight matrices) when specific bit patterns of hidden unit activations are
obtained. In order to better control for the overfitting that might result, we
propose a parametrization that is tree-structured, where each node of the tree
corresponds to a prefix of a sequence of sign bits, or gating units, associated
with hidden units.

该论文提出了一种新颖的神经网络权重矩阵参数化方法，通过激活一些参数和计算 “按需” 地在每个样本上进行，从而可以提高参数数量与计算之间的比率，以提高深度神经网络的泛化能力，并且通过树状结构的参数化方式，进一步控制过拟合。

深度学习中条件计算的容量与计算比例呈指数级增长

Exponentially Increasing the Capacity-to-Computation Ratio for  Conditional Computation in Deep Learning

We consider processes on social networks that can potentially involve three
factors: homophily, or the formation of social ties due to matching individual
traits; social contagion, also known as social influence; and the causal effect
of an individual's covariates on their behavior or other measurable responses.
We show that, generically, all of these are confounded with each other.
Distinguishing them from one another requires strong assumptions on the
parametrization of the social process or on the adequacy of the covariates used
(or both). In particular we demonstrate, with simple examples, that asymmetries
in regression coefficients cannot identify causal effects, and that very simple
models of imitation (a form of social contagion) can produce substantial
correlations between an individual's enduring traits and their choices, even
when there is no intrinsic affinity between them. We also suggest some possible
constructive responses to these results.

本文考虑社交网络上的过程，重点关注同质性、社会影响和个体协变量对行为的因果效应，并提出这些因素一般都是混杂在一起的，区分它们需要对社会过程的参数化或使用的协量变量的充分性进行强假设。

同质性和传染性在观察性社交网络研究中普遍混淆

Homophily and Contagion Are Generically Confounded in Observational  Social Network Studies

In this paper, we describe centering and noncentering methodology as
complementary techniques for use in parametrization of broad classes of
hierarchical models, with a view to the construction of effective MCMC
algorithms for exploring posterior distributions from these models. We give a
clear qualitative understanding as to when centering and noncentering work
well, and introduce theory concerning the convergence time complexity of Gibbs
samplers using centered and noncentered parametrizations. We give general
recipes for the construction of noncentered parametrizations, including an
auxiliary variable technique called the state-space expansion technique. We
also describe partially noncentered methods, and demonstrate their use in
constructing robust Gibbs sampler algorithms whose convergence properties are
not overly sensitive to the data.

本文介绍使用中心化和非中心化方法对广泛类别的分层模型进行参数化的方法，旨在构建有效的 MCMC 算法以探索后验分布。介绍中心化和非中心化何时效果好，并介绍了使用中心化和非中心化参数化的 Gibbs 采样器的收敛时间复杂度理论。同时，还提供了构造非中心化参数化的通用方案，包括称为状态空间扩展技术的辅助变量技术。还介绍了部分非中心化方法，并演示了它们在构建稳健的 Gibbs 采样器算法中的应用，其收敛性不会过于敏感于数据。