Tensor decomposition is one of the well-known approaches to reduce the
latency time and number of parameters of a pre-trained model. However, in this
paper, we propose an approach to use tensor decomposition to reduce training
time of training a model from scratch. In our approach, we train the model from
scratch (i.e., randomly initialized weights) with its original architecture for
a small number of epochs, then the model is decomposed, and then continue
training the decomposed model till the end. There is an optional step in our
approach to convert the decomposed architecture back to the original
architecture. We present results of using this approach on both CIFAR10 and
Imagenet datasets, and show that there can be upto 2x speed up in training time
with accuracy drop of upto 1.5% only, and in other cases no accuracy drop. This
training acceleration approach is independent of hardware and is expected to
have similar speed ups on both CPU and GPU platforms.

本文提出一种使用张量分解减少模型训练时间的方法，通过在原始架构下训练模型一小段时间后对其进行分解，在没有准确度损失或仅有 1.5％的准确度降低的同时训练速度提高了最多 2 倍，而且在 CPU 和 GPU 平台上均可使用。