Increasing effort is put into the development of methods for learning
mechanistic models from data. This task entails not only the accurate
estimation of parameters, but also a suitable model structure. Recent work on
the discovery of dynamical systems formulates this problem as a linear equation
system. Here, we explore several simulation-based optimization approaches,
which allow much greater freedom in the objective formulation and weaker
conditions on the available data. We show that even for relatively small
stochastic population models, simultaneous estimation of parameters and
structure poses major challenges for optimization procedures. Particularly, we
investigate the application of the local stochastic gradient descent method,
commonly used for training machine learning models. We demonstrate accurate
estimation of models but find that enforcing the inference of parsimonious,
interpretable models drastically increases the difficulty. We give an outlook
on how this challenge can be overcome.

从数据中学习机制模型的方法的发展是一个持续的努力，本文通过探索基于模拟的优化方法，讨论了参数估计与结构推断的挑战。

通过梯度下降学习随机人口模型

Towards Learning Stochastic Population Models by Gradient Descent

In federated distributed learning, the goal is to optimize a global training
objective defined over distributed devices, where the data shard at each device
is sampled from a possibly different distribution (a.k.a., heterogeneous or non
i.i.d. data samples). In this paper, we generalize the local stochastic and
full gradient descent with periodic averaging-- originally designed for
homogeneous distributed optimization, to solve nonconvex optimization problems
in federated learning. Although scant research is available on the
effectiveness of local SGD in reducing the number of communication rounds in
homogeneous setting, its convergence and communication complexity in
heterogeneous setting is mostly demonstrated empirically and lacks through
theoretical understating. To bridge this gap, we demonstrate that by properly
analyzing the effect of unbiased gradients and sampling schema in federated
setting, under mild assumptions, the implicit variance reduction feature of
local distributed methods generalize to heterogeneous data shards and exhibits
the best known convergence rates of homogeneous setting both in general
nonconvex and under {\pl}~ condition (generalization of strong-convexity). Our
theoretical results complement the recent empirical studies that demonstrate
the applicability of local GD/SGD to federated learning. We also specialize the
proposed local method for networked distributed optimization. To the best of
our knowledge, the obtained convergence rates are the sharpest known to date on
the convergence of local decant methods with periodic averaging for solving
nonconvex federated optimization in both centralized and networked distributed
optimization.

本文旨在研究在异构样本上进行非凸优化的联邦分布式学习，具体而言，我们将分析分布式方法相对于均匀样本中的隐含方差减少特性在异构样本中的应用，并证明其在广义的非凸和条件下的收敛性与最优性.