This paper describes the architecture and systems built towards solving the
SemEval 2023 Task 2: MultiCoNER II (Multilingual Complex Named Entity
Recognition) [1]. We evaluate two approaches (a) a traditional Conditional
Random Fields model and (b) a Large Language Model (LLM) fine-tuned with a
customized head and compare the two approaches. The novel ideas explored are:
1) Decaying auxiliary loss (with residual) - where we train the model on an
auxiliary task of Coarse-Grained NER and include this task as a part of the
loss function 2) Triplet token blending - where we explore ways of blending the
embeddings of neighboring tokens in the final NER layer prior to prediction 3)
Task-optimal heads - where we explore a variety of custom heads and learning
rates for the final layer of the LLM. We also explore multiple LLMs including
GPT-3 and experiment with a variety of dropout and other hyperparameter
settings before arriving at our final model which achieves micro & macro f1 of
0.85/0.84 (on dev) and 0.67/0.61 on the test data . We show that while
pre-trained LLMs, by themselves, bring about a large improvement in scores as
compared to traditional models, we also demonstrate that tangible improvements
to the Macro-F1 score can be made by augmenting the LLM with additional
feature/loss/model engineering techniques described above.

本文描述了解决 SemEval 2023 Task 2：MultiCoNER II（多语言复杂命名实体识别）问题所构建的体系结构和系统。我们评估了两种方法：传统的条件随机场模型和经过自定义头部微调的大型语言模型（LLM），并比较了这两种方法。我们探索的新想法有：1）衰减辅助损失（带剩余项）- 在模型上训练粗粒度 NER 的辅助任务并将其包括在损失函数的一部分中；2）三元标记混合 - 探索在最终 NER 层中预测之前混合相邻标记的嵌入方式；3）任务最优头部 - 探索各种用于 LLM 最终层的自定义头部和学习率。我们还尝试了多个 LLM，包括 GPT-3，并在最终模型中使用了多种 dropout 和其他超参数设置，该模型在开发数据上达到了 0.85/0.84 的微观和宏观 F1 值，并在测试数据上达到了 0.67/0.61。我们证明，虽然预训练的 LLM 本身相对传统模型带来了很大的改进，但通过上述额外特征 / 损失 / 模型工程技术的增强，我们还可以显著提高宏观 F1 分数。