Many theories of deep learning have shown that a deep network can require
dramatically fewer resources to represent a given function compared to a
shallow network. But a question remains: can these efficient representations be
learned using current deep learning techniques? In this work, we test whether
standard deep learning methods can in fact find the efficient representations
posited by several theories of deep representation. Specifically, we train deep
neural networks to learn two simple functions with known efficient solutions:
the parity function and the fast Fourier transform. We find that using
gradient-based optimization, a deep network does not learn the parity function,
unless initialized very close to a hand-coded exact solution. We also find that
a deep linear neural network does not learn the fast Fourier transform, even in
the best-case scenario of infinite training data, unless the weights are
initialized very close to the exact hand-coded solution. Our results suggest
that not every element of the class of compositional functions can be learned
efficiently by a deep network, and further restrictions are necessary to
understand what functions are both efficiently representable and learnable.

本研究测试了标准深度学习方法是否能够发现几种深层表示理论所提出的有效表示，并发现并非深度网络类中的每一个组件都能够被有效地学习，因此需要进一步限制来理解哪些函数既可以有效地表示也可以被学习。

高效深度表示学习能否实现？

Are Efficient Deep Representations Learnable?

A standard approach to Collaborative Filtering (CF), i.e. prediction of user
ratings on items, relies on Matrix Factorization techniques. Representations
for both users and items are computed from the observed ratings and used for
prediction. Unfortunatly, these transductive approaches cannot handle the case
of new users arriving in the system, with no known rating, a problem known as
user cold-start. A common approach in this context is to ask these incoming
users for a few initialization ratings. This paper presents a model to tackle
this twofold problem of (i) finding good questions to ask, (ii) building
efficient representations from this small amount of information. The model can
also be used in a more standard (warm) context. Our approach is evaluated on
the classical CF problem and on the cold-start problem on four different
datasets showing its ability to improve baseline performance in both cases.

本研究提出一种模型，在用户冷启动情况下，通过提问找到一些初始化评分，并用这些信息建立有效的代表，以解决协同过滤问题。这种方法在四个不同数据集上得到验证，并改善了基准性能。