Existing work in document-level neural machine translation commonly
concatenates several consecutive sentences as a pseudo-document, and then
learns inter-sentential dependencies. This strategy limits the model's ability
to leverage information from distant context. We overcome this limitation with
a novel Document Flattening (DocFlat) technique that integrates Flat-Batch
Attention (FBA) and Neural Context Gate (NCG) into Transformer model to utilize
information beyond the pseudo-document boundaries. FBA allows the model to
attend to all the positions in the batch and learns the relationships between
positions explicitly and NCG identifies the useful information from the distant
context. We conduct comprehensive experiments and analyses on three benchmark
datasets for English-German translation, and validate the effectiveness of two
variants of DocFlat. Empirical results show that our approach outperforms
strong baselines with statistical significance on BLEU, COMET and accuracy on
the contrastive test set. The analyses highlight that DocFlat is highly
effective in capturing the long-range information.

本文提出了一种文档展开技术（DocFlat），通过整合扁平批量注意力（FBA）和神经上下文门（NCG）到 Transformer 模型中，使其能够利用超出伪文档边界的信息，实现捕捉长距离信息，并在三个基准数据集上进行了全面实验和分析，验证了 DocFlat 及其变体的有效性，并显示出较强的性能。

文档展平：基于文档级神经机器翻译上下文联接的拓展

Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translation

Graph Neural Networks (GNNs) have become a prominent approach to machine
learning with graphs and have been increasingly applied in a multitude of
domains. Nevertheless, since most existing GNN models are based on flat
message-passing mechanisms, two limitations need to be tackled: (i) they are
costly in encoding long-range information spanning the graph structure; (ii)
they are failing to encode features in the high-order neighbourhood in the
graphs as they only perform information aggregation across the observed edges
in the original graph. To deal with these two issues, we propose a novel
Hierarchical Message-passing Graph Neural Networks framework. The key idea is
generating a hierarchical structure that re-organises all nodes in a flat graph
into multi-level super graphs, along with innovative intra- and inter-level
propagation manners. The derived hierarchy creates shortcuts connecting
far-away nodes so that informative long-range interactions can be efficiently
accessed via message passing and incorporates meso- and macro-level semantics
into the learned node representations. We present the first model to implement
this framework, termed Hierarchical Community-aware Graph Neural Network
(HC-GNN), with the assistance of a hierarchical community detection algorithm.
The theoretical analysis illustrates HC-GNN's remarkable capacity in capturing
long-range information without introducing heavy additional computation
complexity. Empirical experiments conducted on 9 datasets under transductive,
inductive, and few-shot settings exhibit that HC-GNN can outperform
state-of-the-art GNN models in network analysis tasks, including node
classification, link prediction, and community detection. Moreover, the model
analysis further demonstrates HC-GNN's robustness facing graph sparsity and the
flexibility in incorporating different GNN encoders.

提出了一种新的 Hierarchical Message-passing Graph Neural Networks 框架，通过生成 hierarchical structure，并采用 inra- 和 inter-level propagation manners 以及 hierarchical community detection algorithm，实现了对长程信息的高效访问，从而在节点分类、社区检测等任务中优于现有基于 GNN 的模型。