In comparison to single-document summarization, abstractive Multi-Document
Summarization (MDS) brings challenges on the representation and coverage of its
lengthy and linked sources. This study develops a Parallel Hierarchical
Transformer (PHT) with attention alignment for MDS. By incorporating word- and
paragraph-level multi-head attentions, the hierarchical architecture of PHT
allows better processing of dependencies at both token and document levels. To
guide the decoding towards a better coverage of the source documents, the
attention-alignment mechanism is then introduced to calibrate beam search with
predicted optimal attention distributions. Based on the WikiSum data, a
comprehensive evaluation is conducted to test improvements on MDS by the
proposed architecture. By better handling the inner- and cross-document
information, results in both ROUGE and human evaluation suggest that our
hierarchical model generates summaries of higher quality relative to other
Transformer-based baselines at relatively low computational cost.

本研究提出了一种带有 Attention Alignment 机制的 Parallel Hierarchical Transformer（PHT）用于 Multi-Document Summarization（MDS）。 通过将单词级和段落级的 multi-head attention 结构综合起来，PHT 的分层体系结构可以更好地处理令牌和文档级别的依赖关系，从而更好地处理内部和跨文档信息。 基于 WikiSum 数据，对所提出的体系结构进行了全面评估，结果表明，相对于其他基于 Transformer 的基线，我们的分层模型可以以相对较低的计算成本，通过更好的处理 MDS 来生成更高质量的摘要。

层次化并行变压器与注意力对齐技术用于抽象多文档摘要

Parallel Hierarchical Transformer with Attention Alignment for Abstractive Multi-Document Summarization

Recent trends in neural network based text-to-speech/speech synthesis
pipelines have employed recurrent Seq2seq architectures that can synthesize
realistic sounding speech directly from text characters. These systems however
have complex architectures and takes a substantial amount of time to train. We
introduce several modifications to these Seq2seq architectures that allow for
faster training time, and also allows us to reduce the complexity of the model
architecture at the same time. We show that our proposed model can achieve
attention alignment much faster than previous architectures and that good audio
quality can be achieved with a model that's much smaller in size. Sample audio
available at this https URL

本研究介绍对 Seq2seq 架构的修改，以提高神经网络基础的文本到语音 / 语音合成流程的训练速度和模型的性能，同时保证合成语音的音质。

Seq2Seq 模型的深度语音合成系统

Deep Text-to-Speech System with Seq2Seq Model

In this paper, we make two contributions to unsupervised domain adaptation
(UDA) using the convolutional neural network (CNN). First, our approach
transfers knowledge in all the convolutional layers through attention
alignment. Most previous methods align high-level representations, e.g.,
activations of the fully connected (FC) layers. In these methods, however, the
convolutional layers which underpin critical low-level domain knowledge cannot
be updated directly towards reducing domain discrepancy. Specifically, we
assume that the discriminative regions in an image are relatively invariant to
image style changes. Based on this assumption, we propose an attention
alignment scheme on all the target convolutional layers to uncover the
knowledge shared by the source domain. Second, we estimate the posterior label
distribution of the unlabeled data for target network training. Previous
methods, which iteratively update the pseudo labels by the target network and
refine the target network by the updated pseudo labels, are vulnerable to label
estimation errors. Instead, our approach uses category distribution to
calculate the cross-entropy loss for training, thereby ameliorating the error
accumulation of the estimated labels. The two contributions allow our approach
to outperform the state-of-the-art methods by +2.6% on the Office-31 dataset.

本文提出了一种使用注意力对齐和后验标签分布估计的卷积神经网络自适应无监督域适应的方法，在 Office-31 数据集上超过其他最先进的方法 2.6％。

深度对抗式注意力对齐：无监督域自适应的优势 —— 以目标期望最大化为例

Deep Adversarial Attention Alignment for Unsupervised Domain Adaptation:  the Benefit of Target Expectation Maximization

In this paper, we improve the attention or alignment accuracy of neural
machine translation by utilizing the alignments of training sentence pairs. We
simply compute the distance between the machine attentions and the "true"
alignments, and minimize this cost in the training procedure. Our experiments
on large-scale Chinese-to-English task show that our model improves both
translation and alignment qualities significantly over the large-vocabulary
neural machine translation system, and even beats a state-of-the-art
traditional syntax-based system.

本文提出利用训练句子对齐来提高神经机器翻译的关注或对齐准确性，通过将机器关注与 “真实” 对齐之间的距离计算，并在训练过程中最小化此代价，我们的实验证明，相对于大型词汇神经机器翻译系统，我们的模型显著提高了翻译和对齐质量，超越了最先进的传统基于语法的系统。