Accurate classification of multi-modal financial documents, containing text,
tables, charts, and images, is crucial but challenging. Traditional text-based
approaches often fail to capture the complex multi-modal nature of these
documents. We propose FinEmbedDiff, a cost-effective vector sampling method
that leverages pre-trained multi-modal embedding models to classify financial
documents. Our approach generates multi-modal embedding vectors for documents,
and compares new documents with pre-computed class embeddings using vector
similarity measures. Evaluated on a large dataset, FinEmbedDiff achieves
competitive classification accuracy compared to state-of-the-art baselines
while significantly reducing computational costs. The method exhibits strong
generalization capabilities, making it a practical and scalable solution for
real-world financial applications.

我们提出了一种成本效益的向量取样方法 FinEmbedDiff，利用预训练的多模态嵌入模型对金融文件进行分类，实现了与最先进基线相媲美的分类准确性，并显著降低了计算成本。该方法具备强大的泛化能力，是解决实际金融应用中多模态文件分类问题的可行且可扩展的解决方案。