Performance modelling of a deep learning application is essential to improve
and quantify the efficiency of the model framework. However, existing
performance models are mostly case-specific, with limited capability for the
new deep learning frameworks/applications. In this paper, we propose a generic
performance model of an application in a distributed environment with a generic
expression of the application execution time that considers the influence of
both intrinsic factors/operations (e.g. algorithmic parameters/internal
operations) and extrinsic scaling factors (e.g. the number of processors, data
chunks and batch size). We formulate it as a global optimization problem and
solve it using regularization on a cost function and differential evolution
algorithm to find the best-fit values of the constants in the generic
expression to match the experimentally determined computation time. We have
evaluated the proposed model on three deep learning frameworks (i.e.,
TensorFlow, MXnet, and Pytorch). The experimental results show that the
proposed model can provide accurate performance predictions and
interpretability. In addition, the proposed work can be applied to any
distributed deep neural network without instrumenting the code and provides
insight into the factors affecting performance and scalability.

本文提出一种针对分布式环境下深度学习应用程序的性能建模方法，使用正则化和差分进化算法解决全局优化问题，可用于任何分布式深度神经网络，为性能和可扩展性提供有帮助的因素解释和准确性预测。