Cross-lingual named entity recognition (NER) aims to train an NER model for
the target language leveraging only labeled source language data and unlabeled
target language data. Prior approaches either perform label projection on
translated source language data or employ a source model to assign pseudo
labels for target language data and train a target model on these
pseudo-labeled data to generalize to the target language. However, these
automatic labeling procedures inevitably introduce noisy labels, thus leading
to a performance drop. In this paper, we propose a Global-Local Denoising
framework (GLoDe) for cross-lingual NER. Specifically, GLoDe introduces a
progressive denoising strategy to rectify incorrect pseudo labels by leveraging
both global and local distribution information in the semantic space. The
refined pseudo-labeled target language data significantly improves the model's
generalization ability. Moreover, previous methods only consider improving the
model with language-agnostic features, however, we argue that target
language-specific features are also important and should never be ignored. To
this end, we employ a simple auxiliary task to achieve this goal. Experimental
results on two benchmark datasets with six target languages demonstrate that
our proposed GLoDe significantly outperforms current state-of-the-art methods.

跨语言命名实体识别领域的一项研究，提出了一种全局 - 局部去噪框架（GLoDe），通过利用语义空间中的全局和局部分布信息，改善了模型的泛化能力。在两个基准数据集上的实验结果表明，GLoDe 显著优于目前最先进的方法。

跨语言命名实体识别全局局部去噪框架的伪标签改进

Improving Pseudo Labels with Global-Local Denoising Framework for  Cross-lingual Named Entity Recognition

Recent research suggests that the feed-forward module within Transformers can
be viewed as a collection of key-value memories, where the keys learn to
capture specific patterns from the input based on the training examples. The
values then combine the output from the 'memories' of the keys to generate
predictions about the next token. This leads to an incremental process of
prediction that gradually converges towards the final token choice near the
output layers. This interesting perspective raises questions about how
multilingual models might leverage this mechanism. Specifically, for
autoregressive models trained on two or more languages, do all neurons (across
layers) respond equally to all languages? No! Our hypothesis centers around the
notion that during pretraining, certain model parameters learn strong
language-specific features, while others learn more language-agnostic (shared
across languages) features. To validate this, we conduct experiments utilizing
parallel corpora of two languages that the model was initially pretrained on.
Our findings reveal that the layers closest to the network's input or output
tend to exhibit more language-specific behaviour compared to the layers in the
middle.

通过分析 Transformer 中的前向模块，研究表明其可以被视为一系列键值记忆，提出了关于多语言模型中神经元对不同语言的响应不平等的假设，并通过实验证实了此假设。

揭示 Transformer 模型中的多语言性：探索前向网络中的语言特征

Unveiling Multilinguality in Transformer Models: Exploring Language  Specificity in Feed-Forward Networks

In the paper, we propose a novel way of improving named entity recognition in
the Korean language using its language-specific features. While the field of
named entity recognition has been studied extensively in recent years, the
mechanism of efficiently recognizing named entities in Korean has hardly been
explored. This is because the Korean language has distinct linguistic
properties that prevent models from achieving their best performances.
Therefore, an annotation scheme for {Korean corpora} by adopting the CoNLL-U
format, which decomposes Korean words into morphemes and reduces the ambiguity
of named entities in the original segmentation that may contain functional
morphemes such as postpositions and particles, is proposed herein. We
investigate how the named entity tags are best represented in this
morpheme-based scheme and implement an algorithm to convert word-based {and
syllable-based Korean corpora} with named entities into the proposed
morpheme-based format. Analyses of the results of {statistical and neural}
models reveal that the proposed morpheme-based format is feasible, and the
{varied} performances of the models under the influence of various additional
language-specific features are demonstrated. Extrinsic conditions were also
considered to observe the variance of the performances of the proposed models,
given different types of data, including the original segmentation and
different types of tagging formats.

本文利用特殊语言特征提高韩语命名实体识别，使用 CoNLL-U 格式提出韩语语料库标注方案，实现命名实体标记在分词中减少歧义，并探讨以词素为基础的标记方案的可行性和性能。

基于语言特征的韩语命名实体识别

Korean Named Entity Recognition Based on Language-Specific Features

We train one multilingual model for dependency parsing and use it to parse
sentences in several languages. The parsing model uses (i) multilingual word
clusters and embeddings; (ii) token-level language information; and (iii)
language-specific features (fine-grained POS tags). This input representation
enables the parser not only to parse effectively in multiple languages, but
also to generalize across languages based on linguistic universals and
typological similarities, making it more effective to learn from limited
annotations. Our parser's performance compares favorably to strong baselines in
a range of data scenarios, including when the target language has a large
treebank, a small treebank, or no treebank for training.

使用多语言模型进行依赖解析，通过使用多语言单词聚类和嵌入，令解析器能够在多种语言中进行有效解析，同时基于语言普遍规律和类型相似性实现跨语言推断，从而使其能够更有效地从有限的注释中进行学习。