Lexical complexity prediction (LCP) is the task of predicting the complexity
of words in a text on a continuous scale. It plays a vital role in simplifying
or annotating complex words to assist readers. To study lexical complexity in
Japanese, we construct the first Japanese LCP dataset. Our dataset provides
separate complexity scores for Chinese/Korean annotators and others to address
the readers' L1-specific needs. In the baseline experiment, we demonstrate the
effectiveness of a BERT-based system for Japanese LCP.

构建了针对日语的第一个 LCP 数据集，并通过 BERT 模型在基准实验中证明了其在日语 LCP 任务中的有效性，同时提供了适合中文 / 韩文标注者和其他读者的分离复杂度得分以满足读者的母语特定需求。

非母语读者的日语词汇复杂性：一个新的数据集

Japanese Lexical Complexity for Non-Native Readers: A New Dataset

The occurrence of unknown words in texts significantly hinders reading
comprehension. To improve accessibility for specific target populations,
computational modelling has been applied to identify complex words in texts and
substitute them for simpler alternatives. In this paper, we present an overview
of computational approaches to lexical complexity prediction focusing on the
work carried out on English data. We survey relevant approaches to this problem
which include traditional machine learning classifiers (e.g. SVMs, logistic
regression) and deep neural networks as well as a variety of features, such as
those inspired by literature in psycholinguistics as well as word frequency,
word length, and many others. Furthermore, we introduce readers to past
competitions and available datasets created on this topic. Finally, we include
brief sections on applications of lexical complexity prediction, such as
readability and text simplification, together with related studies on languages
other than English.

本文主要介绍了如何利用计算模型思想来预测英文文本的词汇复杂度，并详细论述了基于传统机器学习分类器和深度神经网络的不同方法，以及相关实验和应用领域如文本简化。

词汇复杂度预测：概述

Lexical Complexity Prediction: An Overview

Complex word identification (CWI) is a cornerstone process towards proper
text simplification. CWI is highly dependent on context, whereas its difficulty
is augmented by the scarcity of available datasets which vary greatly in terms
of domains and languages. As such, it becomes increasingly more difficult to
develop a robust model that generalizes across a wide array of input examples.
In this paper, we propose a novel training technique for the CWI task based on
domain adaptation to improve the target character and context representations.
This technique addresses the problem of working with multiple domains, inasmuch
as it creates a way of smoothing the differences between the explored datasets.
Moreover, we also propose a similar auxiliary task, namely text simplification,
that can be used to complement lexical complexity prediction. Our model obtains
a boost of up to 2.42% in terms of Pearson Correlation Coefficients in contrast
to vanilla training techniques, when considering the CompLex from the Lexical
Complexity Prediction 2021 dataset. At the same time, we obtain an increase of
3% in Pearson scores, while considering a cross-lingual setup relying on the
Complex Word Identification 2018 dataset. In addition, our model yields
state-of-the-art results in terms of Mean Absolute Error.

本文提出一种基于领域适应的复杂词识别训练技术，以改善目标字符和上下文的表示方式，并建议使用文本简化作为补充的词汇复杂度预测技术，从而在多种语料库和领域上提高了模型精度，并获得了最新平均绝对误差结果。

跨语言和多领域单语环境下的复杂单词识别领域自适应

Domain Adaptation in Multilingual and Multi-Domain Monolingual Settings for Complex Word Identification

This paper presents the results and main findings of SemEval-2021 Task 1 -
Lexical Complexity Prediction. We provided participants with an augmented
version of the CompLex Corpus (Shardlow et al 2020). CompLex is an English
multi-domain corpus in which words and multi-word expressions (MWEs) were
annotated with respect to their complexity using a five point Likert scale.
SemEval-2021 Task 1 featured two Sub-tasks: Sub-task 1 focused on single words
and Sub-task 2 focused on MWEs. The competition attracted 198 teams in total,
of which 54 teams submitted official runs on the test data to Sub-task 1 and 37
to Sub-task 2.

本文介绍了 SemEval-2021 任务 1 的结果和主要发现，该任务专注于单词和多词表达式的词汇复杂性预测，使用 CompLex 语料库对这些单词和表达式进行了标注。该比赛吸引了 198 支队伍参加，其中 54 支队伍提交了关于单词的正式结果，37 支队伍提交了关于多词表达式的正式结果。