Practical Bayesian learning often requires (1) online inference, (2) dynamic
models, and (3) ensembling over multiple different models. Recent advances have
shown how to use random feature approximations to achieve scalable, online
ensembling of Gaussian processes with desirable theoretical properties and
fruitful applications. One key to these methods' success is the inclusion of a
random walk on the model parameters, which makes models dynamic. We show that
these methods can be generalized easily to any basis expansion model and that
using alternative basis expansions, such as Hilbert space Gaussian processes,
often results in better performance. To simplify the process of choosing a
specific basis expansion, our method's generality also allows the ensembling of
several entirely different models, for example, a Gaussian process and
polynomial regression. Finally, we propose a novel method to ensemble static
and dynamic models together.

利用随机特征逼近实现可伸缩、在线的高斯过程集成学习方法可广泛应用于不同的基础扩展模型，并能更好地选择具体的基础扩展方式。同时，能够将多种完全不同的模型进行集成，包括高斯过程和多项式回归。最后，提出了一种新颖的将静态和动态模型进行集成的方法。

动态在线基函数扩展集成

Dynamic Online Ensembles of Basis Expansions

Despite their widespread success, the application of deep neural networks to
functional data remains scarce today. The infinite dimensionality of functional
data means standard learning algorithms can be applied only after appropriate
dimension reduction, typically achieved via basis expansions. Currently, these
bases are chosen a priori without the information for the task at hand and thus
may not be effective for the designated task. We instead propose to adaptively
learn these bases in an end-to-end fashion. We introduce neural networks that
employ a new Basis Layer whose hidden units are each basis functions themselves
implemented as a micro neural network. Our architecture learns to apply
parsimonious dimension reduction to functional inputs that focuses only on
information relevant to the target rather than irrelevant variation in the
input function. Across numerous classification/regression tasks with functional
data, our method empirically outperforms other types of neural networks, and we
prove that our approach is statistically consistent with low generalization
error. Code is available at: https://github.com/jwyyy/AdaFNN.

尽管深度神经网络广泛应用于许多领域，但在数学函数数据分析领域却非常少见。本文提出了自适应学习基函数来实现是否有效的降维，引入神经网络，证明本方法在多种分类 / 回归任务中的实验结果均优于其他类型的神经网络。

自适应基层功能数据分析的深度学习

Deep Learning for Functional Data Analysis with Adaptive Basis Layers

In this paper we accomplish the development of the fast rank-adaptive solver
for tensor-structured symmetric positive definite linear systems in higher
dimensions. In [arXiv:1301.6068] this problem is approached by alternating
minimization of the energy function, which we combine with steps of the basis
expansion in accordance with the steepest descent algorithm. In this paper we
combine the same steps in such a way that the resulted algorithm works with one
or two neighboring cores at a time. The recurrent interpretation of the
algorithm allows to prove the global convergence and to estimate the
convergence rate. We also propose several strategies, both rigorous and
heuristic, to compute new subspaces for the basis enrichment in a more
efficient way. We test the algorithm on a number of high-dimensional problems,
including the non-symmetrical Fokker-Planck and chemical master equations, for
which the efficiency of the method is not fully supported by the theory. In all
examples we observe a convincing fast convergence and high efficiency of the
proposed method.

本文提出了一种快速的基于张量结构的对称正定线性系统求解方法，该方法采用能量函数交替最小化与基函数扩展步骤并结合最陡下降算法，进而实现了全局收敛和高效率求解。