This paper investigates a new online learning problem with doubly-streaming
data, where the data streams are described by feature spaces that constantly
evolve, with new features emerging and old features fading away. The challenges
of this problem are two folds: 1) Data samples ceaselessly flowing in may carry
shifted patterns over time, requiring learners to update hence adapt
on-the-fly. 2) Newly emerging features are described by very few samples,
resulting in weak learners that tend to make error predictions. A plausible
idea to overcome the challenges is to establish relationship between the
pre-and-post evolving feature spaces, so that an online learner can leverage
the knowledge learned from the old features to better the learning performance
on the new features. Unfortunately, this idea does not scale up to
high-dimensional media streams with complex feature interplay, which suffers an
tradeoff between onlineness (biasing shallow learners) and
expressiveness(requiring deep learners). Motivated by this, we propose a novel
OLD^3S paradigm, where a shared latent subspace is discovered to summarize
information from the old and new feature spaces, building intermediate feature
mapping relationship. A key trait of OLD^3S is to treat the model capacity as a
learnable semantics, yields optimal model depth and parameters jointly, in
accordance with the complexity and non-linearity of the input data streams in
an online fashion. Both theoretical analyses and empirical studies substantiate
the viability and effectiveness of our proposal.

本文提出了 OLD^3S 框架，实现在线学习的高效处理，针对在不断更新的特征空间上进行的数据流的学习问题。通过建立新旧特征空间之间的关系，发现一个共享的潜在子空间以总结旧和新特征空间的信息，构建中间特征映射关系，从而将模型容量作为可学习的语义，以在线方式在模型深度和参数之间达到最优平衡。