We investigate semi-structured document classification in a zero-shot
setting. Classification of semi-structured documents is more challenging than
that of standard unstructured documents, as positional, layout, and style
information play a vital role in interpreting such documents. The standard
classification setting where categories are fixed during both training and
testing falls short in dynamic environments where new document categories could
potentially emerge. We focus exclusively on the zero-shot setting where
inference is done on new unseen classes. To address this task, we propose a
matching-based approach that relies on a pairwise contrastive objective for
both pretraining and fine-tuning. Our results show a significant boost in Macro
F$_1$ from the proposed pretraining step in both supervised and unsupervised
zero-shot settings.

本研究使用了匹配方法，并对零样本学习中的半结构化文档分类任务进行了探索，使用对位对比目标来加强监督和无监督的零样本学习，显著提高了宏 F1 的结果。

对比训练提高半结构化文档零样本分类能力

Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents

The 'macro F1' metric is frequently used to evaluate binary, multi-class and
multi-label classification problems. Yet, we find that there exist two
different formulas to calculate this quantity. In this note, we show that only
under rare circumstances the two computations can be considered equivalent.
More specifically, one formula well 'rewards' classifiers which produce a
skewed error type distribution. In fact, the difference in outcome of the two
computations can be as high as 0.5. The two computations may not only diverge
in their scalar result but can also lead to different classifier rankings.

研究比较了两种计算 “宏 F1” 度量的公式，发现只有在稀有情况下这两种计算方法才可以被视为等价的，主要由于一种公式更重视产生偏向错误类型分布的分类器，这两种计算方法不仅在标量结果上会产生不同的结果，还会导致不同的分类器排名。