In this study, we explore the application of transformer-based models for
emotion classification on text data. We train and evaluate several pre-trained
transformer models, on the Emotion dataset using different variants of
transformers. The paper also analyzes some factors that in-fluence the
performance of the model, such as the fine-tuning of the transformer layer, the
trainability of the layer, and the preprocessing of the text data. Our analysis
reveals that commonly applied techniques like removing punctuation and stop
words can hinder model performance. This might be because transformers strength
lies in understanding contextual relationships within text. Elements like
punctuation and stop words can still convey sentiment or emphasis and removing
them might disrupt this context.

本研究探讨了基于 Transformer 模型在文本数据情感分类上的应用。我们训练和评估了几种预训练 Transformer 模型，并对 Transformer 层的微调、可训练性以及文本数据预处理等因素进行了分析。我们的分析发现，常用的技术，如去除标点符号和停用词，可能会影响模型的性能。这可能是因为 Transformer 模型的优势在于理解文本中的上下文关系，而标点符号和停用词仍然可以传达情感或强调，去除它们可能会破坏这种上下文。

基于 Transformer 的情感检测：一项比较研究

Emotion Detection with Transformers: A Comparative Study

An initial procedure in text-as-data applications is text preprocessing. One
of the typical steps, which can substantially facilitate computations, consists
in removing infrequent words believed to provide limited information about the
corpus. Despite popularity of vocabulary pruning, not many guidelines on how to
implement it are available in the literature. The aim of the paper is to fill
this gap by examining the effects of removing infrequent words for the quality
of topics estimated using Latent Dirichlet Allocation. The analysis is based on
Monte Carlo experiments taking into account different criteria for infrequent
terms removal and various evaluation metrics. The results indicate that pruning
is beneficial and that the share of vocabulary which might be eliminated can be
quite considerable.

研究论文通过模拟实验，考察了删除不常见词汇对使用潜在狄利克雷分配估计主题质量的影响，结果表明剪枝是有益的，并且可以消除相当大比例的词汇。

移除低频词对 LDA 模型主题质量的影响分析

Analysing the Impact of Removing Infrequent Words on Topic Quality in  LDA Models

Text preprocessing is often the first step in the pipeline of a Natural
Language Processing (NLP) system, with potential impact in its final
performance. Despite its importance, text preprocessing has not received much
attention in the deep learning literature. In this paper we investigate the
impact of simple text preprocessing decisions (particularly tokenizing,
lemmatizing, lowercasing and multiword grouping) on the performance of a
standard neural text classifier. We perform an extensive evaluation on standard
benchmarks from text categorization and sentiment analysis. While our
experiments show that a simple tokenization of input text is generally
adequate, they also highlight significant degrees of variability across
preprocessing techniques. This reveals the importance of paying attention to
this usually-overlooked step in the pipeline, particularly when comparing
different models. Finally, our evaluation provides insights into the best
preprocessing practices for training word embeddings.

本文研究了文本预处理决策（特别是分词、词形还原、小写和多词组合）对标准神经文本分类器性能的影响，并在文本分类和情感分析这两个方面进行了广泛的评估。我们发现，虽然简单的分词通常足够，但预处理技术之间仍存在显着差异，因此这个通常被忽视的步骤很重要，特别是在比较不同模型时，最终评估提供了有关训练词嵌入的最佳预处理实践的见解。