The behavior of neural networks still remains opaque, and a recently widely
noted phenomenon is that networks often achieve similar performance when
initialized with different random parameters. This phenomenon has attracted
significant attention in measuring the similarity between features learned by
distinct networks. However, feature similarity could be vague in describing the
same feature since equivalent features hardly exist. In this paper, we expand
the concept of equivalent feature and provide the definition of what we call
functionally equivalent features. These features produce equivalent output
under certain transformations. Using this definition, we aim to derive a more
intrinsic metric for the so-called feature complexity regarding the redundancy
of features learned by a neural network at each layer. We offer a formal
interpretation of our approach through the lens of category theory, a
well-developed area in mathematics. To quantify the feature complexity, we
further propose an efficient algorithm named Iterative Feature Merging. Our
experimental results validate our ideas and theories from various perspectives.
We empirically demonstrate that the functionally equivalence widely exists
among different features learned by the same neural network and we could reduce
the number of parameters of the network without affecting the performance.The
IFM shows great potential as a data-agnostic model prune method. We have also
drawn several interesting empirical findings regarding the defined feature
complexity.

神经网络中的功能等价特征通过等价输出定义了特征复杂度，使用迭代特征合并算法可以减少网络参数而不影响性能。

超越神经网络特征相似性：网络特征复杂性及其基于范畴论的解释

Going Beyond Neural Network Feature Similarity: The Network Feature  Complexity and Its Interpretation Using Category Theory

We propose a novel machine learning framework for solving optimization
problems governed by large-scale partial differential equations (PDEs) with
high-dimensional random parameters. Such optimization under uncertainty (OUU)
problems may be computational prohibitive using classical methods, particularly
when a large number of samples is needed to evaluate risk measures at every
iteration of an optimization algorithm, where each sample requires the solution
of an expensive-to-solve PDE. To address this challenge, we propose a new
neural operator approximation of the PDE solution operator that has the
combined merits of (1) accurate approximation of not only the map from the
joint inputs of random parameters and optimization variables to the PDE state,
but also its derivative with respect to the optimization variables, (2)
efficient construction of the neural network using reduced basis architectures
that are scalable to high-dimensional OUU problems, and (3) requiring only a
limited number of training data to achieve high accuracy for both the PDE
solution and the OUU solution. We refer to such neural operators as multi-input
reduced basis derivative informed neural operators (MR-DINOs). We demonstrate
the accuracy and efficiency our approach through several numerical experiments,
i.e. the risk-averse control of a semilinear elliptic PDE and the steady state
Navier--Stokes equations in two and three spatial dimensions, each involving
random field inputs. Across the examples, MR-DINOs offer $10^{3}$--$10^{7}
\times$ reductions in execution time, and are able to produce OUU solutions of
comparable accuracies to those from standard PDE based solutions while being
over $10 \times$ more cost-efficient after factoring in the cost of
construction.

提出一种基于神经网络和基函数的新型算法来求解随机参量的大规模偏微分方程优化问题。通过构建一种新的神经算符逼近偏微分方程的解算符，该算符由减小基构建而成，在保持精度的同时大大缩小了训练数据量和计算成本，并成功用于数值实验中。

基于导数信息的神经算子在高维度不确定性下的高效偏微分方程约束优化

Efficient PDE-Constrained optimization under high-dimensional  uncertainty using derivative-informed neural operators

This paper studies an infinite horizon optimal control problem for
discrete-time linear systems and quadratic criteria, both with random
parameters which are independent and identically distributed with respect to
time. A classical approach is to solve an algebraic Riccati equation that
involves mathematical expectations and requires certain statistical information
of the parameters. In this paper, we propose an online iterative algorithm in
the spirit of Q-learning for the situation where only one random sample of
parameters emerges at each time step. The first theorem proves the equivalence
of three properties: the convergence of the learning sequence, the
well-posedness of the control problem, and the solvability of the algebraic
Riccati equation. The second theorem shows that the adaptive feedback control
in terms of the learning sequence stabilizes the system as long as the control
problem is well-posed. Numerical examples are presented to illustrate our
results.

本文针对离散时间线性系统和二次标准的随机参数情况，提出一种基于 Q-learning 精神的在线迭代算法来求解这个无限时间视角下的最优控制问题。第一定理证明了学习序列的收敛性、控制问题的良态性和代数 Riccati 方程的解的可解性三个属性的等价性。第二定理证明了在控制问题得到良态的前提下，学习序列的自适应反馈控制可以稳定系统。数值例子用于说明我们算法的可行性及有效性。