Domain generalization (DG) aims to enhance the model robustness against
domain shifts without accessing target domains. A prevalent category of methods
for DG is data augmentation, which focuses on generating virtual samples to
simulate domain shifts. However, existing augmentation techniques in DG are
mainly tailored for convolutional neural networks (CNNs), with limited
exploration in token-based architectures, i.e., vision transformer (ViT) and
multi-layer perceptrons (MLP) models. In this paper, we study the impact of
prior CNN-based augmentation methods on token-based models, revealing their
performance is suboptimal due to the lack of incentivizing the model to learn
holistic shape information. To tackle the issue, we propose the SEmantic-aware
Token Augmentation (SETA) method. SETA transforms token features by perturbing
local edge cues while preserving global shape features, thereby enhancing the
model learning of shape information. To further enhance the generalization
ability of the model, we introduce two stylized variants of our method combined
with two state-of-the-art style augmentation methods in DG. We provide a
theoretical insight into our method, demonstrating its effectiveness in
reducing the generalization risk bound. Comprehensive experiments on five
benchmarks prove that our method achieves SOTA performances across various ViT
and MLP architectures. Our code is available at
this https URL

通过引入 SEmantic-aware Token Augmentation (SETA) 方法，并结合两种现有样式增强方法，我们在多个视觉转换器 (ViT) 和多层感知器 (MLP) 架构上实现了同类方法中表现最佳的跨域泛化性能。

SETA：针对领域泛化的语义感知令牌增强

SETA: Semantic-Aware Token Augmentation for Domain Generalization

Argument Unit Recognition and Classification aims at identifying argument
units from text and classifying them as pro or against. One of the design
choices that need to be made when developing systems for this task is what the
unit of classification should be: segments of tokens or full sentences.
Previous research suggests that fine-tuning language models on the token-level
yields more robust results for classifying sentences compared to training on
sentences directly. We reproduce the study that originally made this claim and
further investigate what exactly token-based systems learned better compared to
sentence-based ones. We develop systematic tests for analysing the behavioural
differences between the token-based and the sentence-based system. Our results
show that token-based models are generally more robust than sentence-based
models both on manually perturbed examples and on specific subpopulations of
the data.

本文旨在识别和分类文本中的论证单位并将其归类为支持或反对。研究表明，基于 token 的模型可以比基于句子的模型更好地分类句子，所以本文复制并进一步探究了这一结论及其原因，并进行了系统测试以分析使用这两种模型的行为差异，结果表明使用基于 token 的模型通常比基于句子的模型更加健壮。

基于记号的论点单元识别鲁棒性测试中的摄动和子群

Perturbations and Subpopulations for Testing Robustness in Token-Based Argument Unit Recognition

Type- and token-based embedding architectures are still competing in lexical
semantic change detection. The recent success of type-based models in
SemEval-2020 Task 1 has raised the question why the success of token-based
models on a variety of other NLP tasks does not translate to our field. We
investigate the influence of a range of variables on clusterings of BERT
vectors and show that its low performance is largely due to orthographic
information on the target word, which is encoded even in the higher layers of
BERT representations. By reducing the influence of orthography we considerably
improve BERT's performance.

本文考察了 token-based 与 type-based 嵌入模型在词汇语义变化检测中的竞争，研究发现 BERT 表现不佳的根本原因在于目标词汇的构字信息被编码在 BERT 表示的高层中，该论文通过减少构字信息的影响成功地改善了 BERT 的性能。