We propose a generative flow-induced neural architecture search algorithm.
The proposed approach devices simple feed-forward neural networks to learn
stochastic policies to generate sequences of architecture hyperparameters such
that the generated states are in proportion with the reward from the terminal
state. We demonstrate the efficacy of the proposed search algorithm on the
wavelet neural operator (WNO), where we learn a policy to generate a sequence
of hyperparameters like wavelet basis and activation operators for wavelet
integral blocks. While the trajectory of the generated wavelet basis and
activation sequence is cast as flow, the policy is learned by minimizing the
flow violation between each state in the trajectory and maximizing the reward
from the terminal state. In the terminal state, we train WNO simultaneously to
guide the search. We propose to use the exponent of the negative of the WNO
loss on the validation dataset as the reward function. While the grid
search-based neural architecture generation algorithms foresee every
combination, the proposed framework generates the most probable sequence based
on the positive reward from the terminal state, thereby reducing exploration
time. Compared to reinforcement learning schemes, where complete episodic
training is required to get the reward, the proposed algorithm generates the
hyperparameter trajectory sequentially. Through four fluid mechanics-oriented
problems, we illustrate that the learned policies can sample the
best-performing architecture of the neural operator, thereby improving the
performance of the vanilla wavelet neural operator.

我们提出了一种生成的流诱导神经架构搜索算法，使用简单的前馈神经网络学习随机策略生成超参数序列，通过最小化流违规和最大化终态奖励来训练架构搜索策略，在流之中生成了小波基础和激活序列，并通过流的生成路径来提高神经算子性能。