Recent works have demonstrated reasonable success of representation learning
in hypercomplex space. Specifically, "fully-connected layers with Quaternions"
(4D hypercomplex numbers), which replace real-valued matrix multiplications in
fully-connected layers with Hamilton products of Quaternions, both enjoy
parameter savings with only 1/4 learnable parameters and achieve comparable
performance in various applications. However, one key caveat is that
hypercomplex space only exists at very few predefined dimensions (4D, 8D, and
16D). This restricts the flexibility of models that leverage hypercomplex
multiplications. To this end, we propose parameterizing hypercomplex
multiplications, allowing models to learn multiplication rules from data
regardless of whether such rules are predefined. As a result, our method not
only subsumes the Hamilton product, but also learns to operate on any arbitrary
nD hypercomplex space, providing more architectural flexibility using
arbitrarily $1/n$ learnable parameters compared with the fully-connected layer
counterpart. Experiments of applications to the LSTM and Transformer models on
natural language inference, machine translation, text style transfer, and
subject verb agreement demonstrate architectural flexibility and effectiveness
of the proposed approach.

提出了一种参数化超复数乘法的方法，使得模型可以从数据中学习乘法规则，进而在任意维度的超复数空间上进行运算，从而获得更大的架构灵活性和性能优势。