Stein Variational Gradient Descent (SVGD) can transport particles along
trajectories that reduce the KL divergence between the target and particle
distribution but requires the target score function to compute the update. We
introduce a new perspective on SVGD that views it as a local estimator of the
reversed KL gradient flow. This perspective inspires us to propose new
estimators that use local linear models to achieve the same purpose. The
proposed estimators can be computed using only samples from the target and
particle distribution without needing the target score function. Our proposed
variational gradient estimators utilize local linear models, resulting in
computational simplicity while maintaining effectiveness comparable to SVGD in
terms of estimation biases. Additionally, we demonstrate that under a mild
assumption, the estimation of high-dimensional gradient flow can be translated
into a lower-dimensional estimation problem, leading to improved estimation
accuracy. We validate our claims with experiments on both simulated and
real-world datasets.

通过使用本地线性模型，提出了一种使用目标和粒子分布的样本计算梯度的新方法，可以替代需要计算目标得分函数的 Stein 变分梯度下降（SVGD）方法。

使用本地线性模型的变分梯度下降

Variational Gradient Descent using Local Linear Models

A common pipeline in learning-based control is to iteratively estimate a
model of system dynamics, and apply a trajectory optimization algorithm -
e.g.~$\mathtt{iLQR}$ - on the learned model to minimize a target cost. This
paper conducts a rigorous analysis of a simplified variant of this strategy for
general nonlinear systems. We analyze an algorithm which iterates between
estimating local linear models of nonlinear system dynamics and performing
$\mathtt{iLQR}$-like policy updates. We demonstrate that this algorithm attains
sample complexity polynomial in relevant problem parameters, and, by
synthesizing locally stabilizing gains, overcomes exponential dependence in
problem horizon. Experimental results validate the performance of our
algorithm, and compare to natural deep-learning baselines.

本文对于一种学习控制策略进行了系统分析，该策略主要包括估计系统动态模型和应用轨迹优化算法来降低目标成本，我们提出了一种基于本地线性模型的算法，在重要的问题参数上获得了多项式的样本复杂度，并通过合成本地稳定增益，克服了问题时间影响的指数依赖性，我们的实验结果也验证了该算法的有效性并与自然深度学习基线进行了比较。

非线性策略优化中学习本地线性模型的威力

The Power of Learned Locally Linear Models for Nonlinear Policy  Optimization

The dynamics of complex systems generally include high-dimensional,
non-stationary and non-linear behavior, all of which pose fundamental
challenges to quantitative understanding. To address these difficulties we
detail a new approach based on local linear models within windows determined
adaptively from the data. While the dynamics within each window are simple,
consisting of exponential decay, growth and oscillations, the collection of
local parameters across all windows provides a principled characterization of
the full time series. To explore the resulting model space, we develop a novel
likelihood-based hierarchical clustering and we examine the eigenvalues of the
linear dynamics. We demonstrate our analysis with the Lorenz system undergoing
stable spiral dynamics and in the standard chaotic regime. Applied to the
posture dynamics of the nematode $C. elegans$ our approach identifies
fine-grained behavioral states and model dynamics which fluctuate close to an
instability boundary, and we detail a bifurcation in a transition from forward
to backward crawling. Finally, we analyze whole-brain imaging in $C. elegans$
and show that the stability of global brain states changes with oxygen
concentration.

利用局部线性模型和层次聚类的方法，对复杂系统进行分析，将数据划分为多个窗口进行简单的指数衰减和振荡分析，通过收集每个窗口的参数，提取出时间序列的特征，从而可以应用于分析 $C. elegans$ 神经元运动和全脑成像等研究。