We present an approach based on feed-forward neural networks for learning the
distribution of textual documents. This approach is inspired by the Neural
Autoregressive Distribution Estimator(NADE) model, which has been shown to be a
good estimator of the distribution of discrete-valued igh-dimensional vectors.
In this paper, we present how NADE can successfully be adapted to the case of
textual data, retaining from NADE the property that sampling or computing the
probability of observations can be done exactly and efficiently. The approach
can also be used to learn deep representations of documents that are
competitive to those learned by the alternative topic modeling approaches.
Finally, we describe how the approach can be combined with a regular neural
network N-gram model and substantially improve its performance, by making its
learned representation sensitive to the larger, document-specific context.

本文提出了一种基于前馈神经网络的方法来学习文本文件的分布，可以用来学习文本数据的深度表示，并结合一个常规神经网络 N-gram 模型来显著提高性能。

文档神经自回归分布估计

Document Neural Autoregressive Distribution Estimation

In recent years there is a growing interest in using deep representations for
reinforcement learning. In this paper, we present a methodology and tools to
analyze Deep Q-networks (DQNs) in a non-blind matter. Moreover, we propose a
new model, the Semi Aggregated Markov Decision Process (SAMDP), and an
algorithm that learns it automatically. The SAMDP model allows us to identify
spatio-temporal abstractions directly from features and may be used as a
sub-goal detector in future work. Using our tools we reveal that the features
learned by DQNs aggregate the state space in a hierarchical fashion, explaining
its success. Moreover, we are able to understand and describe the policies
learned by DQNs for three different Atari2600 games and suggest ways to
interpret, debug and optimize deep neural networks in reinforcement learning.

本文介绍了分析 Deep Q-networks（DQNs）的一种方法和工具，以及自动学习 Semi Aggregated Markov Decision Process（SAMDP）模型的算法。SAMDP 模型允许我们直接从特征中识别时空抽象，并且可以在今后的工作中用作子目标检测器。使用我们的工具，我们揭示了 DQNs 学习的特征以层次方式聚合状态空间，解释了其成功。此外，我们能够理解和描述 DQNs 为三个不同的 Atari2600 游戏学习的策略，并提出解释、调试和优化强化学习中深度神经网络的方式。

灰盒子：理解 DQNs

Graying the black box: Understanding DQNs

In recent years there have been many successes of using deep representations
in reinforcement learning. Still, many of these applications use conventional
architectures, such as convolutional networks, LSTMs, or auto-encoders. In this
paper, we present a new neural network architecture for model-free
reinforcement learning. Our dueling network represents two separate estimators:
one for the state value function and one for the state-dependent action
advantage function. The main benefit of this factoring is to generalize
learning across actions without imposing any change to the underlying
reinforcement learning algorithm. Our results show that this architecture leads
to better policy evaluation in the presence of many similar-valued actions.
Moreover, the dueling architecture enables our RL agent to outperform the
state-of-the-art on the Atari 2600 domain.

本文介绍一种新的神经网络架构，即竞争网络，用于无模型强化学习。竞争网络表示两个独立的估计器：状态值函数和状态依赖性动作优势函数。结果表明，竞争架构有助于在类似值的动作存在的情况下实现更好的策略评估，并使得强化学习代理能够在 Atari 2600 领域超越现有技术。

深度强化学习的竞技网络架构

Dueling Network Architectures for Deep Reinforcement Learning

We consider a set of probabilistic functions of some input variables as a
representation of the inputs. We present bounds on how informative a
representation is about input data. We extend these bounds to hierarchical
representations so that we can quantify the contribution of each layer towards
capturing the information in the original data. The special form of these
bounds leads to a simple, bottom-up optimization procedure to construct
hierarchical representations that are also maximally informative about the
data. This optimization has linear computational complexity and constant sample
complexity in the number of variables. These results establish a new approach
to unsupervised learning of deep representations that is both principled and
practical. We demonstrate the usefulness of the approach on both synthetic and
real-world data.

本研究介绍了一种基于概率函数的输入变量表示方法，并提出如何量化各层对原始数据信息提取的贡献，并应用于深度表示的无监督学习，具有可实施性和科学性。