Recurrent neural network architectures combining with attention mechanism, or
neural attention model, have shown promising performance recently for the tasks
including speech recognition, image caption generation, visual question
answering and machine translation. In this paper, neural attention model is
applied on two sequence classification tasks, dialogue act detection and key
term extraction. In the sequence labeling tasks, the model input is a sequence,
and the output is the label of the input sequence. The major difficulty of
sequence labeling is that when the input sequence is long, it can include many
noisy or irrelevant part. If the information in the whole sequence is treated
equally, the noisy or irrelevant part may degrade the classification
performance. The attention mechanism is helpful for sequence classification
task because it is capable of highlighting important part among the entire
sequence for the classification task. The experimental results show that with
the attention mechanism, discernible improvements were achieved in the sequence
labeling task considered here. The roles of the attention mechanism in the
tasks are further analyzed and visualized in this paper.

本文研究探讨了应用于对话行为检测和关键词提取的神经注意力模型在序列分类任务中的应用和作用，并通过实验结果和可视化分析验证了该模型在去除噪音和提取关键信息方面的表现。

适用于序列分类的神经注意力模型：分析及其在关键术语提取和对话行为检测中的应用

Neural Attention Models for Sequence Classification: Analysis and  Application to Key Term Extraction and Dialogue Act Detection

We propose a reparameterization of LSTM that brings the benefits of batch
normalization to recurrent neural networks. Whereas previous works only apply
batch normalization to the input-to-hidden transformation of RNNs, we
demonstrate that it is both possible and beneficial to batch-normalize the
hidden-to-hidden transition, thereby reducing internal covariate shift between
time steps. We evaluate our proposal on various sequential problems such as
sequence classification, language modeling and question answering. Our
empirical results show that our batch-normalized LSTM consistently leads to
faster convergence and improved generalization.

本研究提出一种 LSTM 重新参数化的方法，可以将批归一化的优点引入到循环神经网络中，通过对隐层之间的转换进行批归一化，减少时间步长之间的内部协变量漂移，在序列分类、语言建模和问答系统等问题中验证了方法的有效性和实用性。

循环批量归一化

Recurrent Batch Normalization

The proliferation of sensor devices monitoring human activity generates
voluminous amount of temporal sequences needing to be interpreted and
categorized. Moreover, complex behavior detection requires the personalization
of multi-sensor fusion algorithms. Conditional random fields (CRFs) are
commonly used in structured prediction tasks such as part-of-speech tagging in
natural language processing. Conditional probabilities guide the choice of each
tag/label in the sequence conflating the structured prediction task with the
sequence classification task where different models provide different
categorization of the same sequence. The claim of this paper is that CRF models
also provide discriminative models to distinguish between types of sequence
regardless of the accuracy of the labels obtained if we calibrate the class
membership estimate of the sequence. We introduce and compare different neural
network based linear-chain CRFs and we present experiments on two complex
sequence classification and structured prediction tasks to support this claim.

该研究论文探讨了如何使用基于条件随机场（CRFs）的多传感器融合算法对人类活动监控的大量时间序列进行分类，同时提供了区分不同类型序列的判别模型，实验结果证明了该方法的有效性。