Transformer based pre-trained models such as BERT and its variants, which are
trained on large corpora, have demonstrated tremendous success for natural
language processing (NLP) tasks. Most of academic works are based on the
English language; however, the number of multilingual and language specific
studies increase steadily. Furthermore, several studies claimed that language
specific models outperform multilingual models in various tasks. Therefore, the
community tends to train or fine-tune the models for the language of their case
study, specifically. In this paper, we focus on Turkish maps data and
thoroughly evaluate both multilingual and Turkish based BERT, DistilBERT,
ELECTRA and RoBERTa. Besides, we also propose a MultiLayer Perceptron (MLP) for
fine-tuning BERT in addition to the standard approach of one-layer fine-tuning.
For the dataset, a mid-sized Address Parsing corpus taken with a relatively
high quality is constructed. Conducted experiments on this dataset indicate
that Turkish language specific models with MLP fine-tuning yields slightly
better results when compared to the multilingual fine-tuned models. Moreover,
visualization of address tokens' representations further indicates the
effectiveness of BERT variants for classifying a variety of addresses.

本文研究了使用 Transformer 模型 Bert、DistilBert、ELECTRA 和 RoBERTa，以及多层感知器 MLP 对土耳其地图数据进行多语言和土耳其语 fine-tune 的实验，结果表明土耳其语特定模型与多语言 fine-tuning 的相比略微更好，并且 BERT 变种对于地址类别分类的效果非常好。