We propose a Regularized Adaptive Momentum Dual Averaging (RAMDA) algorithm
for training structured neural networks. Similar to existing regularized
adaptive methods, the subproblem for computing the update direction of RAMDA
involves a nonsmooth regularizer and a diagonal preconditioner, and therefore
does not possess a closed-form solution in general. We thus also carefully
devise an implementable inexactness condition that retains convergence
guarantees similar to the exact versions, and propose a companion efficient
solver for the subproblems of both RAMDA and existing methods to make them
practically feasible. We leverage the theory of manifold identification in
variational analysis to show that, even in the presence of such inexactness,
the iterates of RAMDA attain the ideal structure induced by the regularizer at
the stationary point of asymptotic convergence. This structure is locally
optimal near the point of convergence, so RAMDA is guaranteed to obtain the
best structure possible among all methods converging to the same point, making
it the first regularized adaptive method outputting models that possess
outstanding predictive performance while being (locally) optimally structured.
Extensive numerical experiments in large-scale modern computer vision, language
modeling, and speech tasks show that the proposed RAMDA is efficient and
consistently outperforms state of the art for training structured neural
network. Implementation of our algorithm is available at
this http URL

我们提出了一种用于训练结构化神经网络的正则化自适应动量双重平均（RAMDA）算法，通过引入一个有效求解子问题的方法，RAMDA 可以输出具有优秀预测性能和（局部）最佳结构的模型。

结构化神经网络训练的正则化自适应动量双平均方法和高效非精确子问题求解器

Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact  Subproblem Solver for Training Structured Neural Network

The paper proposes the use of structured neural networks for reinforcement
learning based nonlinear adaptive control. The focus is on partially observable
systems, with separate neural networks for the state and feedforward observer
and the state feedback and feedforward controller. The observer dynamics are
modelled by recurrent neural networks while a standard network is used for the
controller. As discussed in the paper, this leads to a separation of the
observer dynamics to the recurrent neural network part, and the state feedback
to the feedback and feedforward network. The structured approach reduces the
computational complexity and gives the reinforcement learning based controller
an {\em understandable} structure as compared to when one single neural network
is used. As shown by simulation the proposed structure has the additional and
main advantage that the training becomes significantly faster. Two ways to
include feedforward structure are presented, one related to state feedback
control and one related to classical feedforward control. The latter method
introduces further structure with a separate recurrent neural network that
processes only the measured disturbance. When evaluated with simulation on a
nonlinear cascaded double tank process, the method with most structure performs
the best, with excellent feedforward disturbance rejection gains.

本文提出使用结构化神经网络进行基于强化学习的非线性自适应控制，聚焦于部分可观测系统和前馈结构，将观测器动态建模为循环神经网络，采用标准网络用于控制器。这种结构化方法在降低计算复杂性的同时，提供了可理解的控制结构，并在模拟中得到了显著的训练速度提升。

强化学习中的观察器 - 反馈 - 前馈控制结构

Observer-Feedback-Feedforward Controller Structures in Reinforcement  Learning

We propose the Poisson neural networks (PNNs) to learn Poisson systems and
trajectories of autonomous systems from data. Based on the Darboux-Lie theorem,
the phase flow of a Poisson system can be written as the composition of (1) a
coordinate transformation, (2) an extended symplectic map and (3) the inverse
of the transformation. In this work, we extend this result to the unknotted
trajectories of autonomous systems. We employ structured neural networks with
physical priors to approximate the three aforementioned maps. We demonstrate
through several simulations that PNNs are capable of handling very accurately
several challenging tasks, including the motion of a particle in the
electromagnetic potential, the nonlinear Schr{\"o}dinger equation, and pixel
observations of the two-body problem.

本文提出了泊松神经网络（PNN），使用结构化的神经网络来近似三种映射，从而学习泊松系统和自治系统的轨迹，模拟证明了 PNN 可以准确地处理包括粒子在电磁势中的运动、非线性薛定谔方程和二体问题的像素观察等若干具有挑战性的任务。