We propose a combined three pre-trained language models (XLM-R, BART, and
DeBERTa-V3) as an empower of contextualized embedding for named entity
recognition. Our model achieves a 92.9% F1 score on the test set and ranks 5th
on the leaderboard at NL4Opt competition subtask 1.

本文使用三个预训练的语言模型（XLM-R、BART 和 DeBERTa-V3）作为上下文嵌入的增强，用于命名实体识别，最终在 NL4Opt 竞赛子任务 1 上取得了 92.9％ 的 F1 得分，并排名第五。

VTCC-NLP 参加 NL4Opt 竞赛子任务 1：命名实体识别的集成预训练语言模型

VTCC-NLP at NL4Opt competition subtask 1: An Ensemble Pre-trained language models for Named Entity Recognition

We describe a new addition to the WebVectors toolkit which is used to serve
word embedding models over the Web. The new ELMoViz module adds support for
contextualized embedding architectures, in particular for ELMo models. The
provided visualizations follow the metaphor of `two-dimensional text' by
showing lexical substitutes: words which are most semantically similar in
context to the words of the input sentence. The system allows the user to
change the ELMo layers from which token embeddings are inferred. It also
conveys corpus information about the query words and their lexical substitutes
(namely their frequency tiers and parts of speech). The module is well
integrated into the rest of the WebVectors toolkit, providing lexical
hyperlinks to word representations in static embedding models. Two web services
have already implemented the new functionality with pre-trained ELMo models for
Russian, Norwegian and English.

该研究论文介绍了一种名为 WebVectors 的工具包，其中新增了 ELMoViz 模块，将支持上下文嵌入架构，特别是 ELMo 模型，提供的可视化工具为‘二维文本’元喻，展示了语境中语义最相似的词语选项，还支持用户可更改 ELMo 层次，同时传达了查询词及其词汇替代物（即其频率级别和词性）的语料库信息，并且模块已经很好的整合到 WebVectors 工具包中，提供了静态嵌入模型中的词表示的词汇超链接，已有两个 Web 服务实现了俄语、挪威语和英语的预训练 ELMo 模型。

将 ELMo 嵌入表示为二维文本在线

Representing ELMo embeddings as two-dimensional text online

This paper presents new state-of-the-art models for three tasks,
part-of-speech tagging, syntactic parsing, and semantic parsing, using the
cutting-edge contextualized embedding framework known as BERT. For each task,
we first replicate and simplify the current state-of-the-art approach to
enhance its model efficiency. We then evaluate our simplified approaches on
those three tasks using token embeddings generated by BERT. 12 datasets in both
English and Chinese are used for our experiments. The BERT models outperform
the previously best-performing models by 2.5% on average (7.5% for the most
significant case). Moreover, an in-depth analysis on the impact of BERT
embeddings is provided using self-attention, which helps understanding in this
rich yet representation. All models and source codes are available in public so
that researchers can improve upon and utilize them to establish strong
baselines for the next decade.

本研究采用最新的语境化嵌入框架 BERT，为词性标注、句法分析和语义分析三个任务提出了最新的技术模型，并在 12 个英文和中文数据集上进行了实验，结果表明 BERT 模型在这三项任务中均优于当前最佳模型平均 2.5％（最显着的情况下是 7.5％），并提供了关于 BERT 嵌入影响的深入分析。