Recent advances in digitization has led to availability of multivariate time
series data in various domains, in order to monitor operations in real time.
Identifying abnormal data pattern and detect potential failures in these
scenarios are important yet rather difficult tasks. We propose a novel
unsupervised anomaly detection method for time series data. Our approach uses
sequence encoder and decoder to represent the mapping between time series and
hidden state, and learns bidirectional dynamics simultaneously by leveraging
backward and forward temporal information in the training process. We further
regularize the state space to place constraints on states of normal samples,
and use Mahalanobis distance to evaluate abnormality level. Results on
synthetic and real-world datasets show the superiority of the proposed method.

本文提出一种基于序列编码器和解码器的，利用前后向时间信息学习双向动态的无监督异常检测方法，通过对正常样本状态空间的正则化和使用马氏距离评估异常等级，实现对复杂多元时间序列数据中异常数据模式的识别和潜在故障的检测。

基于序列重建的状态空间模型的时间序列异常检测

Time series anomaly detection with sequence reconstruction based state-space model

In this paper, we propose Cross-Thought, a novel approach to pre-training
sequence encoder, which is instrumental in building reusable sequence
embeddings for large-scale NLP tasks such as question answering. Instead of
using the original signals of full sentences, we train a Transformer-based
sequence encoder over a large set of short sequences, which allows the model to
automatically select the most useful information for predicting masked words.
Experiments on question answering and textual entailment tasks demonstrate that
our pre-trained encoder can outperform state-of-the-art encoders trained with
continuous sentence signals as well as traditional masked language modeling
baselines. Our proposed approach also achieves new state of the art on HotpotQA
(full-wiki setting) by improving intermediate information retrieval
performance.

这篇论文提出了 Cross-Thought 方法用以预训练序列编码器，通过大规模的短序列训练 Transformer-based 序列编码器来自动选择对预测掩码词最有用的信息，用于大规模自然语言处理任务，如问答，文本推断等，实验结果表明，所提出的方法比传统基于连续句子信号的最新编码器以及传统掩码语言模型基线更加优秀，并打破了 HotpotQA (full-wiki setting) 的最新记录，取得了新的最高水平的中间信息检索表现。

句子编码器预训练的交叉思维

Cross-Thought for Sentence Encoder Pre-training

Event argument extraction (EAE) aims to identify the arguments of an event
and classify the roles that those arguments play. Despite great efforts made in
prior work, there remain many challenges: (1) Data scarcity. (2) Capturing the
long-range dependency, specifically, the connection between an event trigger
and a distant event argument. (3) Integrating event trigger information into
candidate argument representation. For (1), we explore using unlabeled data in
different ways. For (2), we propose to use a syntax-attending Transformer that
can utilize dependency parses to guide the attention mechanism. For (3), we
propose a trigger-aware sequence encoder with several types of
trigger-dependent sequence representations. We also support argument extraction
either from text annotated with gold entities or from plain text. Experiments
on the English ACE2005 benchmark show that our approach achieves a new
state-of-the-art.

该研究提出了利用语法引导 transformer 的长距离依存关系，来提高事件面向序列编码的实体识别性能。在 ACE2005 数据集上，该方法取得了新的最高性能。