In distributed machine learning, a central node outsources computationally
expensive calculations to external worker nodes. The properties of optimization
procedures like stochastic gradient descent (SGD) can be leveraged to mitigate
the effect of unresponsive or slow workers called stragglers, that otherwise
degrade the benefit of outsourcing the computation. This can be done by only
waiting for a subset of the workers to finish their computation at each
iteration of the algorithm. Previous works proposed to adapt the number of
workers to wait for as the algorithm evolves to optimize the speed of
convergence. In contrast, we model the communication and computation times
using independent random variables. Considering this model, we construct a
novel scheme that adapts both the number of workers and the computation load
throughout the run-time of the algorithm. Consequently, we improve the
convergence speed of distributed SGD while significantly reducing the
computation load, at the expense of a slight increase in communication load.

该研究通过适应节点和运算负载调整方法，提高分布式随机梯度下降算法的收敛速度，显著降低计算负载，但略微增加通信负载。

减少计算负载的快速容错分布式 SGD

Fast and Straggler-Tolerant Distributed SGD with Reduced Computation  Load

This paper presents a computationally efficient and robust LiDAR-inertial
odometry framework. We fuse LiDAR feature points with IMU data using a
tightly-coupled iterated extended Kalman filter to allow robust navigation in
fast-motion, noisy or cluttered environments where degeneration occurs. To
lower the computation load in the presence of large number of measurements, we
present a new formula to compute the Kalman gain. The new formula has
computation load depending on the state dimension instead of the measurement
dimension. The proposed method and its implementation are tested in various
indoor and outdoor environments. In all tests, our method produces reliable
navigation results in real-time: running on a quadrotor onboard computer, it
fuses more than 1,200 effective feature points in a scan and completes all
iterations of an iEKF step within 25 ms. Our codes are open-sourced on Github.

本文提出了一种高效、稳定的激光雷达惯性组合导航算法，它基于迭代扩展卡尔曼滤波，将激光和 IMU 数据进行融合，有效地实现了在快速运动，嘈杂或杂乱环境中的关闭式导航，在大数据处理时具有较低的计算复杂度，并成功测试于室内外的各种场景中，并且非常适合在四轴飞行器等小型计算设备上实施。

FAST-LIO: 一种基于紧耦合迭代卡尔曼滤波的快速、鲁棒的激光雷达 - 惯导组合导航算法包

FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by  Tightly-Coupled Iterated Kalman Filter

This paper develops coding techniques to reduce the running time of
distributed learning tasks. It characterizes the fundamental tradeoff to
compute gradients (and more generally vector summations) in terms of three
parameters: computation load, straggler tolerance and communication cost. It
further gives an explicit coding scheme that achieves the optimal tradeoff
based on recursive polynomial constructions, coding both across data subsets
and vector components. As a result, the proposed scheme allows to minimize the
running time for gradient computations. Implementations are made on Amazon EC2
clusters using Python with mpi4py package. Results show that the proposed
scheme maintains the same generalization error while reducing the running time
by $32\%$ compared to uncoded schemes and $23\%$ compared to prior coded
schemes focusing only on stragglers (Tandon et al., ICML 2017).

该论文提出了一种编码技术，以减少分布式学习任务的运行时间，并基于递归多项式结构提出了一种编码方案，优化了梯度计算的运行时间，同时保持相同泛化误差。