In this work, we measure the impact of affixal negation on modern English
large language models (LLMs). In affixal negation, the negated meaning is
expressed through a negative morpheme, which is potentially challenging for
LLMs as their tokenizers are often not morphologically plausible. We conduct
extensive experiments using LLMs with different subword tokenization methods,
which lead to several insights on the interaction between tokenization
performance and negation sensitivity. Despite some interesting mismatches
between tokenization accuracy and negation detection performance, we show that
models can, on the whole, reliably recognize the meaning of affixal negation.

本文旨在研究测量后缀否定对现代英语大型语言模型（LLMs）的影响。通过负性语素表达被否定的含义，这对 LLMs 来说可能会具有挑战性，因为它们的分词器往往不具备形态的可行性。通过使用不同的子词分词方法，我们进行了大量实验，得出了有关分词性能和否定敏感性之间相互作用的几个见解。尽管分词准确性和否定检测性能之间存在一些有趣的不匹配，但我们证明模型总体上能够可靠地识别后缀否定的含义。

重温次词标记化：大型语言模型中的词缀否定案例研究

Revisiting subword tokenization: A case study on affixal negation in  large language models

The success of models operating on tokenized data has led to an increased
demand for effective tokenization methods, particularly when applied to vision
or auditory tasks, which inherently involve non-discrete data. One of the most
popular tokenization methods is Vector Quantization (VQ), a key component of
several recent state-of-the-art methods across various domains. Typically, a VQ
Variational Autoencoder (VQVAE) is trained to transform data to and from its
tokenized representation. However, since the VQVAE is trained with a
reconstruction objective, there is no constraint for the embeddings to be well
disentangled, a crucial aspect for using them in discriminative tasks.
Recently, several works have demonstrated the benefits of utilizing hyperbolic
spaces for representation learning. Hyperbolic spaces induce compact latent
representations due to their exponential volume growth and inherent ability to
model hierarchical and structured data. In this work, we explore the use of
hyperbolic spaces for vector quantization (HyperVQ), formulating the VQ
operation as a hyperbolic Multinomial Logistic Regression (MLR) problem, in
contrast to the Euclidean K-Means clustering used in VQVAE. Through extensive
experiments, we demonstrate that hyperVQ performs comparably in reconstruction
and generative tasks while outperforming VQ in discriminative tasks and
learning a highly disentangled latent space.

利用超 bolic 空间进行向量量化 (HyperVQ)，通过对比 VQVAE 中的欧几里得 K-Means 聚类，形成了超 bolic 多项式逻辑回归问题的 VQ 操作。通过广泛的实验，我们证明 hyperVQ 在重建和生成任务中的性能相当，而在判别任务中胜过 VQ，并学习高度独立的潜在空间。

HyperVQ: 基于 MLR 的双曲空间向量量化

HyperVQ: MLR-based Vector Quantization in Hyperbolic Space

Nigerian Pidgin is arguably the most widely spoken language in Nigeria.
Variants of this language are also spoken across West and Central Africa,
making it a very important language. This work aims to establish supervised and
unsupervised neural machine translation (NMT) baselines between English and
Nigerian Pidgin. We implement and compare NMT models with different
tokenization methods, creating a solid foundation for future works.

研究实现了监督和无监督神经机器翻译模型来翻译英语和尼日利亚皮钦语，比较了不同的分词方法，在英语和尼日利亚皮钦语之间建立了良好的翻译基础。