We consider automatically identifying the defined term within a mathematical
definition from the text of an academic article. Inspired by the development of
transformer-based natural language processing applications, we pose the problem
as (a) a token-level classification task using fine-tuned pre-trained
transformers; and (b) a question-answering task using a generalist large
language model (GPT). We also propose a rule-based approach to build a labeled
dataset from the LATEX source of papers. Experimental results show that it is
possible to reach high levels of precision and recall using either recent (and
expensive) GPT 4 or simpler pre-trained models fine-tuned on our task.

我们考虑从学术论文的文本中自动识别数学定义中的明确术语。我们将问题作为（a）使用微调的预训练变压器进行令牌级分类任务，以及（b）使用通用大型语言模型（GPT）进行问答任务来解决。我们还提出了一种基于规则的方法，通过从论文的 LATEX 源代码构建一个带标签的数据集。实验结果表明，使用最近（和昂贵的）GPT 4 或在我们的任务上进行微调的较简单的预训练模型均可以达到高精度和召回率的水平。

利用 Transformer 提取数学学术文章中的定义

Extracting Definienda in Mathematical Scholarly Articles with  Transformers

Attention is an operation that selects some largest element from some set,
where the notion of largest is defined elsewhere. Applying this operation to
sequence to sequence mapping results in significant improvements to the task at
hand. In this paper we provide the mathematical definition of attention and
examine its application to sequence to sequence models. We highlight the exact
correspondences between machine learning implementations of attention and our
mathematical definition. We provide clear evidence of effectiveness of
attention mechanisms evaluating models with varying degrees of attention on a
very simple task: copying a sentence. We find that models that make greater use
of attention perform much better on sequence to sequence mapping tasks,
converge faster and are more stable.

本文提供了注意力的数学定义，并研究了其在序列到序列模型中的应用。通过对复制句子等简单任务的评估，得出注意力机制的有效性证据，更多地使用注意力的模型在序列到序列映射任务中表现更好，收敛更快，更稳定。

复制这个句子

Copy this Sentence

Lossy compression algorithms are typically designed and analyzed through the
lens of Shannon's rate-distortion theory, where the goal is to achieve the
lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate.
However, in recent years, it has become increasingly accepted that "low
distortion" is not a synonym for "high perceptual quality", and in fact
optimization of one often comes at the expense of the other. In light of this
understanding, it is natural to seek for a generalization of rate-distortion
theory which takes perceptual quality into account. In this paper, we adopt the
mathematical definition of perceptual quality recently proposed by Blau &
Michaeli (2018), and use it to study the three-way tradeoff between rate,
distortion, and perception. We show that restricting the perceptual quality to
be high, generally leads to an elevation of the rate-distortion curve, thus
necessitating a sacrifice in either rate or distortion. We prove several
fundamental properties of this triple-tradeoff, calculate it in closed form for
a Bernoulli source, and illustrate it visually on a toy MNIST example.

通过采用 Blau & Michaeli 2018 年提出的感知质量的数学定义，研究了速率、失真和感知之间的三重权衡，我们证明了该三重权衡的几个基本属性，并在一个玩具 MNIST 例子上进行了可视化说明。