Whereas deep neural networks were first mostly used for classification tasks,
they are rapidly expanding in the realm of structured output problems, where
the observed target is composed of multiple random variables that have a rich
joint distribution, given the input. We focus in this paper on the case where
the input also has a rich structure and the input and output structures are
somehow related. We describe systems that learn to attend to different places
in the input, for each element of the output, for a variety of tasks: machine
translation, image caption generation, video clip description and speech
recognition. All these systems are based on a shared set of building blocks:
gated recurrent neural networks and convolutional neural networks, along with
trained attention mechanisms. We report on experimental results with these
systems, showing impressively good performance and the advantage of the
attention mechanism.

本文介绍了使用深度神经网络中的递归神经网络和卷积神经网络等措施来实现注意力机制，以解决多元随机变量富含联合分布的结构化输出问题。通过典型任务如机器翻译、图像字幕生成、视频片段描述和语音识别实验，系统展示了很好的性能。

使用基于注意力机制的编码 - 解码网络描述多媒体内容

Describing Multimedia Content using Attention-based Encoder--Decoder  Networks

Conditional Restricted Boltzmann Machines (CRBMs) are rich probabilistic
models that have recently been applied to a wide range of problems, including
collaborative filtering, classification, and modeling motion capture data.
While much progress has been made in training non-conditional RBMs, these
algorithms are not applicable to conditional models and there has been almost
no work on training and generating predictions from conditional RBMs for
structured output problems. We first argue that standard Contrastive
Divergence-based learning may not be suitable for training CRBMs. We then
identify two distinct types of structured output prediction problems and
propose an improved learning algorithm for each. The first problem type is one
where the output space has arbitrary structure but the set of likely output
configurations is relatively small, such as in multi-label classification. The
second problem is one where the output space is arbitrarily structured but
where the output space variability is much greater, such as in image denoising
or pixel labeling. We show that the new learning algorithms can work much
better than Contrastive Divergence on both types of problems.

本文介绍了两种针对结构化输出问题的改进型学习算法：一种适用于输出空间具有任意结构但可能输出配置集较小的问题，另一种适用于输出空间具有任意结构且输出空间的可变性较大的问题。这些新型算法在这两种问题类型上都表现得比对比散度更好。