Linguistic ambiguity continues to represent a significant challenge for
natural language processing (NLP) systems, notwithstanding the advancements in
architectures such as Transformers and BERT. Inspired by the recent success of
instructional models like ChatGPT and Gemini (In 2023, the artificial
intelligence was called Bard.), this study aims to analyze and discuss
linguistic ambiguity within these models, focusing on three types prevalent in
Brazilian Portuguese: semantic, syntactic, and lexical ambiguity. We create a
corpus comprising 120 sentences, both ambiguous and unambiguous, for
classification, explanation, and disambiguation. The models capability to
generate ambiguous sentences was also explored by soliciting sets of sentences
for each type of ambiguity. The results underwent qualitative analysis, drawing
on recognized linguistic references, and quantitative assessment based on the
accuracy of the responses obtained. It was evidenced that even the most
sophisticated models, such as ChatGPT and Gemini, exhibit errors and
deficiencies in their responses, with explanations often providing
inconsistent. Furthermore, the accuracy peaked at 49.58 percent, indicating the
need for descriptive studies for supervised learning.

本研究旨在分析和讨论 ChatGPT 和 Gemini 等模型中的语言歧义，关注巴西葡萄牙语中的语义、句法和词汇歧义。通过创建一个包含 120 个歧义和非歧义句子的语料库，进行分类、解释和消除歧义。结果经过定性分析和基于回答准确性的定量评估，证明了即使是像 ChatGPT 和 Gemini 这样先进的模型在回答中也存在错误和不足，解释通常是不一致的。此外，准确性达到了 49.58％的峰值，表明有必要进行有监督学习的描述性研究。

大规模语言模型（LLMs）中的语义歧义分析

Análise de ambiguidade linguística em modelos de linguagem de grande  escala (LLMs)

Linguistic ambiguity is and has always been one of the main challenges in
Natural Language Processing (NLP) systems. Modern Transformer architectures
like BERT, T5 or more recently InstructGPT have achieved some impressive
improvements in many NLP fields, but there is still plenty of work to do.
Motivated by the uproar caused by ChatGPT, in this paper we provide an
introduction to linguistic ambiguity, its varieties and their relevance in
modern NLP, and perform an extensive empiric analysis. ChatGPT strengths and
weaknesses are revealed, as well as strategies to get the most of this model.

本文介绍语言歧义及其在现代自然语言处理中的相关性，并基于 ChatGPT 做了丰富的实证分析以揭示其优点和缺陷，同时提出最佳实践策略。

ChatGPT 中的语言歧义分析

Linguistic ambiguity analysis in ChatGPT

Natural Language Inference (NLI) datasets contain examples with highly
ambiguous labels. While many research works do not pay much attention to this
fact, several recent efforts have been made to acknowledge and embrace the
existence of ambiguity, such as UNLI and ChaosNLI. In this paper, we explore
the option of training directly on the estimated label distribution of the
annotators in the NLI task, using a learning loss based on this ambiguity
distribution instead of the gold-labels. We prepare AmbiNLI, a trial dataset
obtained from readily available sources, and show it is possible to reduce
ChaosNLI divergence scores when finetuning on this data, a promising first step
towards learning how to capture linguistic ambiguity. Additionally, we show
that training on the same amount of data but targeting the ambiguity
distribution instead of gold-labels can result in models that achieve higher
performance and learn better representations for downstream tasks.

本文研究了在自然语言推理任务中训练模型时，直接使用标注者标签分布的方法而非正确标签的方法，并准备了 AmbiNLI 数据集，经过模型微调可以降低混沌度得分，并可提高模型性能和下游任务的表现。