In many scientific research and engineering applications where repeated
simulations of complex systems are conducted, a surrogate is commonly adopted
to quickly estimate the whole system. To reduce the expensive cost of
generating training examples, it has become a promising approach to combine the
results of low-fidelity (fast but inaccurate) and high-fidelity (slow but
accurate) simulations. Despite the fast developments of multi-fidelity fusion
techniques, most existing methods require particular data structures and do not
scale well to high-dimensional output. To resolve these issues, we generalize
the classic autoregression (AR), which is wildly used due to its simplicity,
robustness, accuracy, and tractability, and propose generalized autoregression
(GAR) using tensor formulation and latent features. GAR can deal with arbitrary
dimensional outputs and arbitrary multifidelity data structure to satisfy the
demand of multi-fidelity fusion for complex problems; it admits a fully
tractable likelihood and posterior requiring no approximate inference and
scales well to high-dimensional problems. Furthermore, we prove the
autokrigeability theorem based on GAR in the multi-fidelity case and develop
CIGAR, a simplified GAR with the exact predictive mean accuracy with
computation reduction by a factor of d 3, where d is the dimensionality of the
output. The empirical assessment includes many canonical PDEs and real
scientific examples and demonstrates that the proposed method consistently
outperforms the SOTA methods with a large margin (up to 6x improvement in RMSE)
with only a couple high-fidelity training samples.

本文提出了基于张量分解以及潜在特征的广义自回归（GAR）方法，在多保真融合问题中能够处理任意维度的输出和任意保真数据结构，并且具有良好的可扩展性和精度，同时提出了简化版的算法 CIGAR 以降低计算复杂度，经过实验验证，该方法明显优于现有的同类方法。

通用自回归用于多保真度融合

GAR: Generalized Autoregression for Multi-Fidelity Fusion

We study the implicit bias of gradient flow (i.e., gradient descent with
infinitesimal step size) on linear neural network training. We propose a tensor
formulation of neural networks that includes fully-connected, diagonal, and
convolutional networks as special cases, and investigate the linear version of
the formulation called linear tensor networks. With this formulation, we can
characterize the convergence direction of the network parameters as singular
vectors of a tensor defined by the network. For $L$-layer linear tensor
networks that are orthogonally decomposable, we show that gradient flow on
separable classification finds a stationary point of the $\ell_{2/L}$
max-margin problem in a "transformed" input space defined by the network. For
underdetermined regression, we prove that gradient flow finds a global minimum
which minimizes a norm-like function that interpolates between weighted
$\ell_1$ and $\ell_2$ norms in the transformed input space. Our theorems
subsume existing results in the literature while removing standard convergence
assumptions. We also provide experiments that corroborate our analysis.

研究了线性神经网络训练中渐进流（即用无穷小步长的梯度下降法）的隐含偏差；提出了神经网络的张量形式，包括全连接、对角线和卷积网络等特例，并研究了称为线性张量网络的公式的线性版本。通过这个公式，我们可以将网络的收敛方向表征为由网络定义的张量的奇异向量。