In this work, we consider the problem of sequence-to-sequence alignment for
signals containing outliers. Assuming the absence of outliers, the standard
Dynamic Time Warping (DTW) algorithm efficiently computes the optimal alignment
between two (generally) variable-length sequences. While DTW is robust to
temporal shifts and dilations of the signal, it fails to align sequences in a
meaningful way in the presence of outliers that can be arbitrarily interspersed
in the sequences. To address this problem, we introduce Drop-DTW, a novel
algorithm that aligns the common signal between the sequences while
automatically dropping the outlier elements from the matching. The entire
procedure is implemented as a single dynamic program that is efficient and
fully differentiable. In our experiments, we show that Drop-DTW is a robust
similarity measure for sequence retrieval and demonstrate its effectiveness as
a training loss on diverse applications. With Drop-DTW, we address temporal
step localization on instructional videos, representation learning from noisy
videos, and cross-modal representation learning for audio-visual retrieval and
localization. In all applications, we take a weakly- or unsupervised approach
and demonstrate state-of-the-art results under these settings.

本文中，我们提出了一种新颖的 Drop-DTW 算法，用于解决包含异常点情况下的序列对齐问题，并将其成功应用于涉及到无监督模型训练和跨模态媒体匹配的多个应用场景中，并取得了最先进的实验结果。