This work proposes the Two-headed DragoNet, a Transformer-based model for
hierarchical multi-label classification of financial transactions. Our model is
based on a stack of Transformers encoder layers that generate contextual
embeddings from two short textual descriptors (merchant name and business
activity), followed by a Context Fusion layer and two output heads that
classify transactions according to a hierarchical two-level taxonomy (macro and
micro categories). Finally, our proposed Taxonomy-aware Attention Layer
corrects predictions that break categorical hierarchy rules defined in the
given taxonomy. Our proposal outperforms classical machine learning methods in
experiments of macro-category classification by achieving an F1-score of 93\%
on a card dataset and 95% on a current account dataset.

本研究提出了一种基于 Transformer 的 Two-headed DragoNet 模型，用于金融交易的分层多标签分类。我们的模型基于一系列 Transformer 编码层，从两个简短的文本描述（商家名称和业务活动）生成上下文嵌入，然后经过一个上下文融合层和两个输出头根据两级分类法（宏观和微观类别）对交易进行分类。最后，我们提出的基于分类法的注意力层修正了违反给定分类法定义的层次规则的预测。在基于卡片的数据集上，我们的提出方法在宏观类别分类的实验中表现优于传统的机器学习方法，达到了 93% 的 F1 分数；在活期账户数据集上达到了 95% 的 F1 分数。