Fully tensorial theory of hypercomplex neural networks is given. The key
point is to observe that the algebra multiplication can be represented as a
rank three tensor. This approach is attractive for neural network libraries
that support effective tensorial operations.

给出了超复杂神经网络的完全张量理论，其关键点是观察代数乘法可以表示为一个三阶张量。这种方法对于支持有效张量操作的神经网络库非常有吸引力。

完全张量法在超复杂神经网络中的应用

Fully tensorial approach to hypercomplex neural networks

Scaling the distributed deep learning to a massive GPU cluster level is
challenging due to the instability of the large mini-batch training and the
overhead of the gradient synchronization. We address the instability of the
large mini-batch training with batch-size control and label smoothing. We
address the overhead of the gradient synchronization with 2D-Torus all-reduce.
Specifically, 2D-Torus all-reduce arranges GPUs in a logical 2D grid and
performs a series of collective operation in different orientations. These two
techniques are implemented with Neural Network Libraries (NNL). We have
successfully trained ImageNet/ResNet-50 in 122 seconds without significant
accuracy loss on ABCI cluster.

使用批次大小控制和标签平滑来解决大批次训练不稳定性问题；使用 2D-Torus all-reduce 解决梯度同步过程中的开销，实现了在 ABC 集群上在 122 秒内训练 ImageNet/ResNet-50 且精度损失不显著的分布式深度学习技术.