This paper serves as a foundational step towards the development of a
linguistically motivated and technically relevant evaluation suite for Greek
NLP. We initiate this endeavor by introducing four expert-verified evaluation
tasks, specifically targeted at natural language inference, word sense
disambiguation (through example comparison or sense selection) and metaphor
detection. More than language-adapted replicas of existing tasks, we contribute
two innovations which will resonate with the broader resource and evaluation
community. Firstly, our inference dataset is the first of its kind, marking not
just \textit{one}, but rather \textit{all} possible inference labels,
accounting for possible shifts due to e.g. ambiguity or polysemy. Secondly, we
demonstrate a cost-efficient method to obtain datasets for under-resourced
languages. Using ChatGPT as a language-neutral parser, we transform the
Dictionary of Standard Modern Greek into a structured format, from which we
derive the other three tasks through simple projections. Alongside each task,
we conduct experiments using currently available state of the art machinery.
Our experimental baselines affirm the challenging nature of our tasks and
highlight the need for expedited progress in order for the Greek NLP ecosystem
to keep pace with contemporary mainstream research.

本论文旨在为希腊自然语言处理（Greek NLP）领域开发一个基于语言学动机和技术相关的评估套件，通过引入四个专家验证的评估任务来实现这一目标，这些任务特别针对自然语言推理、词义消歧（通过示例比较或意义选择）和隐喻检测。同时，我们贡献了两个创新，既是我们推理数据集是首个标注所有可能的推理标签的数据集，也是通过使用 ChatGPT 作为语言中立的解析器以成本效率高的方式获取到面向资源不足语言的数据集。通过对每个任务进行实验，使用现有的最先进技术，我们确认了任务的挑战性，并强调希腊 NLP 生态系统需要加速进展以与当代主流研究保持同步的需求。