Despite their original goal to jointly learn to align and translate, Neural
Machine Translation (NMT) models, especially Transformer, are often perceived
as not learning interpretable word alignments. In this paper, we show that NMT
models do learn interpretable word alignments, which could only be revealed
with proper interpretation methods. We propose a series of such methods that
are model-agnostic, are able to be applied either offline or online, and do not
require parameter update or architectural change. We show that under the force
decoding setup, the alignments induced by our interpretation method are of
better quality than fast-align for some systems, and when performing free
decoding, they agree well with the alignments induced by automatic alignment
tools.

本文探讨了神经机器翻译模型在实现翻译时，虽然目标是学习对齐和翻译，但其对齐质量难以解释。作者提出了一种模型无关的解释方法，能够提高对齐质量。

基于显著性的神经机器翻译词对齐解释

Saliency-driven Word Alignment Interpretation for Neural Machine  Translation

Post-hoc model-agnostic interpretation methods such as partial dependence
plots can be employed to interpret complex machine learning models. While these
interpretation methods can be applied regardless of model complexity, they can
produce misleading and verbose results if the model is too complex, especially
w.r.t. feature interactions. To quantify the complexity of arbitrary machine
learning models, we propose model-agnostic complexity measures based on
functional decomposition: number of features used, interaction strength and
main effect complexity. We show that post-hoc interpretation of models that
minimize the three measures is more reliable and compact. Furthermore, we
demonstrate the application of these measures in a multi-objective optimization
approach which simultaneously minimizes loss and complexity.

提出了基于功能分解的模型不可知复杂度量度来量化任意机器学习模型的复杂度，并展示了在同时最小化损失和复杂度的多目标优化中应用这些量度的方法。

通过功能分解量化模型复杂度，以获得更好的事后解释性

Quantifying Model Complexity via Functional Decomposition for Better  Post-Hoc Interpretability

One way to interpret neural model predictions is to highlight the most
important input features---for example, a heatmap visualization over the words
in an input sentence. In existing interpretation methods for NLP, a word's
importance is determined by either input perturbation---measuring the decrease
in model confidence when that word is removed---or by the gradient with respect
to that word. To understand the limitations of these methods, we use input
reduction, which iteratively removes the least important word from the input.
This exposes pathological behaviors of neural models: the remaining words
appear nonsensical to humans and are not the ones determined as important by
interpretation methods. As we confirm with human experiments, the reduced
examples lack information to support the prediction of any label, but models
still make the same predictions with high confidence. To explain these
counterintuitive results, we draw connections to adversarial examples and
confidence calibration: pathological behaviors reveal difficulties in
interpreting neural models trained with maximum likelihood. To mitigate their
deficiencies, we fine-tune the models by encouraging high entropy outputs on
reduced examples. Fine-tuned models become more interpretable under input
reduction without accuracy loss on regular examples.

通过 input reduction 方法研究了神经网络模型的缺陷，发现在面对异常数据时大多数模型都表现出困难并难以解释，提出了一种 fine-tuning 方法，通过提高模型的输出熵，增强模型的可解释性。