This paper serves as a foundational step towards the development of a
linguistically motivated and technically relevant evaluation suite for Greek
NLP. We initiate this endeavor by introducing four expert-verified evaluation
tasks, specifically targeted at natural language inference, word sense
disambiguation (through example comparison or sense selection) and metaphor
detection. More than language-adapted replicas of existing tasks, we contribute
two innovations which will resonate with the broader resource and evaluation
community. Firstly, our inference dataset is the first of its kind, marking not
just \textit{one}, but rather \textit{all} possible inference labels,
accounting for possible shifts due to e.g. ambiguity or polysemy. Secondly, we
demonstrate a cost-efficient method to obtain datasets for under-resourced
languages. Using ChatGPT as a language-neutral parser, we transform the
Dictionary of Standard Modern Greek into a structured format, from which we
derive the other three tasks through simple projections. Alongside each task,
we conduct experiments using currently available state of the art machinery.
Our experimental baselines affirm the challenging nature of our tasks and
highlight the need for expedited progress in order for the Greek NLP ecosystem
to keep pace with contemporary mainstream research.

本论文旨在为希腊自然语言处理（Greek NLP）领域开发一个基于语言学动机和技术相关的评估套件，通过引入四个专家验证的评估任务来实现这一目标，这些任务特别针对自然语言推理、词义消歧（通过示例比较或意义选择）和隐喻检测。同时，我们贡献了两个创新，既是我们推理数据集是首个标注所有可能的推理标签的数据集，也是通过使用 ChatGPT 作为语言中立的解析器以成本效率高的方式获取到面向资源不足语言的数据集。通过对每个任务进行实验，使用现有的最先进技术，我们确认了任务的挑战性，并强调希腊 NLP 生态系统需要加速进展以与当代主流研究保持同步的需求。

OYXOY：用于现代希腊语的现代自然语言处理测试套件

OYXOY: A Modern NLP Test Suite for Modern Greek

The supervised training of high-capacity models on large datasets containing
hundreds of thousands of document-summary pairs is critical to the recent
success of deep learning techniques for abstractive summarization.
Unfortunately, in most domains (other than news) such training data is not
available and cannot be easily sourced. In this paper we enable the use of
supervised learning for the setting where there are only documents available
(e.g.,~product or business reviews) without ground truth summaries. We create a
synthetic dataset from a corpus of user reviews by sampling a review,
pretending it is a summary, and generating noisy versions thereof which we
treat as pseudo-review input. We introduce several linguistically motivated
noise generation functions and a summarization model which learns to denoise
the input and generate the original review. At test time, the model accepts
genuine reviews and generates a summary containing salient opinions, treating
those that do not reach consensus as noise. Extensive automatic and human
evaluation shows that our model brings substantial improvements over both
abstractive and extractive baselines.

本文介绍了一种基于🈚️ground truth summaries 的情况下，利用文档创建合成数据集、引入多种噪声生成函数以及学习生成原始评论的摘要模型的方法，该方法比抽取式和生成式基线模型都有更好的效果。