In many real-world scenarios, distribution shifts exist in the streaming data
across time steps. Many complex sequential data can be effectively divided into
distinct regimes that exhibit persistent dynamics. Discovering the shifted
behaviors and the evolving patterns underlying the streaming data are important
to understand the dynamic system. Existing methods typically train one robust
model to work for the evolving data of distinct distributions or sequentially
adapt the model utilizing explicitly given regime boundaries. However, there
are two challenges: (1) shifts in data streams could happen drastically and
abruptly without precursors. Boundaries of distribution shifts are usually
unavailable, and (2) training a shared model for all domains could fail to
capture varying patterns. This paper aims to solve the problem of sequential
data modeling in the presence of sudden distribution shifts that occur without
any precursors. Specifically, we design a Bayesian framework, dubbed as T-SaS,
with a discrete distribution-modeling variable to capture abrupt shifts of
data. Then, we design a model that enable adaptation with dynamic network
selection conditioned on that discrete variable. The proposed method learns
specific model parameters for each distribution by learning which neurons
should be activated in the full network. A dynamic masking strategy is adopted
here to support inter-distribution transfer through the overlapping of a set of
sparse networks. Extensive experiments show that our proposed method is
superior in both accurately detecting shift boundaries to get segments of
varying distributions and effectively adapting to downstream forecast or
classification tasks.

在没有任何先兆的情况下，本文旨在解决连续数据建模中突发分布转变的问题。具体而言，我们设计了一个名为 T-SaS 的贝叶斯框架，使用离散分布建模变量捕捉数据的突变。然后，我们设计了一个模型，通过学习判断在完整网络中应该激活哪些神经元来实现与该离散变量的动态网络选择进行自适应。通过一组稀疏网络的重叠，采用动态遮罩策略来支持不同分布之间的互相转移。大量实验证明我们的方法在准确检测分布转变边界以获取不同分布的片段并有效适应下游预测或分类任务方面优越。