To benefit from the modeling capacity of deep models in system
identification, without worrying about inference time, this study presents a
novel training strategy that uses deep models only at the training stage. For
this purpose two separate models with different structures and goals are
employed. The first one is a deep generative model aiming at modeling the
distribution of system output(s), called the teacher model, and the second one
is a shallow basis function model, named the student model, fed by system
input(s) to predict the system output(s). That means these isolated paths must
reach the same ultimate target. As deep models show a great performance in
modeling of highly nonlinear systems, aligning the representation space learned
by these two models make the student model to inherit the approximation power
of the teacher model. The proposed objective function consists of the objective
of each student and teacher model adding up with a distance penalty between the
learned latent representations. The simulation results on three nonlinear
benchmarks show a comparative performance with examined deep architectures
applied on the same benchmarks. Algorithmic transparency and structure
efficiency are also achieved as byproducts.

为了在系统辨识中充分利用深度模型的建模能力，本研究提出了一种新的训练策略，仅在训练阶段使用深度模型。通过采用两个具有不同结构和目标的分离模型，第一个模型是目标为模拟系统输出分布的深度生成模型（教师模型），第二个模型是基于浅层基函数的模型（学生模型），通过系统输入来预测系统输出。通过使这两个模型学习的表示空间保持一致，使学生模型继承了教师模型的逼近能力。仿真结果在三个非线性基准测试上与应用于相同基准测试的深度架构相比表现出相似的性能。同时实现了算法透明性和结构效率作为副产品。