Integrating human feedback in models can improve the performance of natural
language processing (NLP) models. Feedback can be either explicit (e.g. ranking
used in training language models) or implicit (e.g. using human cognitive
signals in the form of eyetracking). Prior eye tracking and NLP research reveal
that cognitive processes, such as human scanpaths, gleaned from human gaze
patterns aid in the understanding and performance of NLP models. However, the
collection of real eyetracking data for NLP tasks is challenging due to the
requirement of expensive and precise equipment coupled with privacy invasion
issues. To address this challenge, we propose ScanTextGAN, a novel model for
generating human scanpaths over text. We show that ScanTextGAN-generated
scanpaths can approximate meaningful cognitive signals in human gaze patterns.
We include synthetically generated scanpaths in four popular NLP tasks spanning
six different datasets as proof of concept and show that the models augmented
with generated scanpaths improve the performance of all downstream NLP tasks.

本文提出了一种基于 ScanTextGAN 的人类注视路径生成模型，通过集成生成的注视路径数据来提高自然语言处理领域中多个任务的性能表现。

为改善 NLP 表现合成人类凝视反馈

Synthesizing Human Gaze Feedback for Improved NLP Performance

Ambiguity is a natural language phenomenon occurring at different levels of
syntax, semantics, and pragmatics. It is widely studied; in Psycholinguistics,
for instance, we have a variety of competing studies for the human
disambiguation processes. These studies are empirical and based on eyetracking
measurements. Here we take first steps towards formalizing these processes for
semantic ambiguities where we identified the presence of two features: (1)
joint plausibility degrees of different possible interpretations, (2) causal
structures according to which certain words play a more substantial role in the
processes. The novel sheaf-theoretic model of definite causality developed by
Gogioso and Pinzani in QPL 2021 offers tools to model and reason about these
features. We applied this theory to a dataset of ambiguous phrases extracted
from Psycholinguistics literature and their human plausibility judgements
collected by us using the Amazon Mechanical Turk engine. We measured the causal
fractions of different disambiguation orders within the phrases and discovered
two prominent orders: from subject to verb in the subject-verb and from object
to verb in the verb object phrases. We also found evidence for delay in the
disambiguation of polysemous vs homonymous verbs, again compatible with
Psycholinguistic findings.

本文提出了一个新的 sheaf 论述模型，用于分析语义歧义的消岐过程。研究表明，在歧义短语中，从主语到动词和从宾语到动词是两种最常见的消岐顺序，并且使用这一模型进行了测试。同时，我们发现多义词和同音异义词的消岐过程存在延迟。

语义歧义的因果结构

The Causal Structure of Semantic Ambiguities

We present a novel approach for determining learners' second language
proficiency which utilizes behavioral traces of eye movements during reading.
Our approach provides stand-alone eyetracking based English proficiency scores
which reflect the extent to which the learner's gaze patterns in reading are
similar to those of native English speakers. We show that our scores correlate
strongly with standardized English proficiency tests. We also demonstrate that
gaze information can be used to accurately predict the outcomes of such tests.
Our approach yields the strongest performance when the test taker is presented
with a suite of sentences for which we have eyetracking data from other
readers. However, it remains effective even using eyetracking with sentences
for which eye movement data have not been previously collected. By deriving
proficiency as an automatic byproduct of eye movements during ordinary reading,
our approach offers a potentially valuable new tool for second language
proficiency assessment. More broadly, our results open the door to future
methods for inferring reader characteristics from the behavioral traces of
reading.

我们提出了一种新颖的方法，利用学习者阅读时的眼动痕迹来确定学习者的第二语言能力水平，并通过其反映出学习者阅读中的凝视模式与母语为英语者的相似程度，这种方法对于第二语言能力评估具有潜在的价值。