There were many algorithms to substitute the back-propagation (BP) in the
deep neural network (DNN) training. However, they could not become popular
because their training accuracy and the computational efficiency were worse
than BP. One of them was direct feedback alignment (DFA), but it showed low
training performance especially for the convolutional neural network (CNN). In
this paper, we overcome the limitation of the DFA algorithm by combining with
the conventional BP during the CNN training. To improve the training stability,
we also suggest the feedback weight initialization method by analyzing the
patterns of the fixed random matrices in the DFA. Finally, we propose the new
training algorithm, binary direct feedback alignment (BDFA) to minimize the
computational cost while maintaining the training accuracy compared with the
DFA. In our experiments, we use the CIFAR-10 and CIFAR-100 dataset to simulate
the CNN learning from the scratch and apply the BDFA to the online learning
based object tracking application to examine the training in the small dataset
environment. Our proposed algorithms show better performance than conventional
BP in both two different training tasks especially when the dataset is small.

本文提出了结合 BP 算法的 BDFA 算法，同时提出了反馈权重初始化方法和模式分析技术来维持训练稳定性和最小化计算成本。 实验结果表明，该方法在小数据集环境下表现优异。