Modern ML systems ingest data aggregated from diverse sources, such as
synthetic, human-annotated, and live customer traffic. Understanding
\textit{which} examples are important to the performance of a learning
algorithm is crucial for efficient model training. Recently, a growing body of
literature has given rise to various "influence scores," which use training
artifacts such as model confidence or checkpointed gradients to identify
important subsets of data. However, these methods have primarily been developed
in computer vision settings, and it remains unclear how well they generalize to
language-based tasks using pretrained models.
In this paper, we explore the applicability of influence scores in language
classification tasks. We evaluate a diverse subset of these scores on the SNLI
dataset by quantifying accuracy changes in response to pruning training data
through random and influence-score-based sampling. We then stress-test one of
the scores -- "variance of gradients" (VoG) from Agarwal et al. (2022) -- in an
NLU model stack that was exposed to dynamic user speech patterns in a voice
assistant type of setting. Our experiments demonstrate that in many cases,
encoder-based language models can be finetuned on roughly 50% of the original
data without degradation in performance metrics. Along the way, we summarize
lessons learned from applying out-of-the-box implementations of influence
scores, quantify the effects of noisy and class-imbalanced data, and offer
recommendations on score-based sampling for better accuracy and training
efficiency.

应用影响力分数评估语言分类任务，通过修剪训练数据来量化准确性变化，并提供基于分数采样的建议，以提高准确性和训练效率。

高效语言数据抽样的规模化影响分数

Influence Scores at Scale for Efficient Language Data Sampling

We present an analysis of the performance of machine learning classifiers on
discriminating between similar languages and language varieties. We carried out
a number of experiments using the results of the two editions of the
Discriminating between Similar Languages (DSL) shared task. We investigate the
progress made between the two tasks, estimate an upper bound on possible
performance using ensemble and oracle combination, and provide learning curves
to help us understand which languages are more challenging. A number of
difficult sentences are identified and investigated further with human
annotation.

本文分析了机器学习分类器在区分相似语言和语言变体方面的表现，使用 DSL 共享任务的两个版本的结果进行了一系列实验，研究了两个任务之间的进展，估计了可能性能的上限，并提供了学习曲线，以帮助我们了解哪些语言更具挑战性。此外，还使用人工注释对一些困难的句子进行了进一步的研究。

辨别相似语言：评估与探索

Discriminating Similar Languages: Evaluations and Explorations

In multilingual question answering, either the question needs to be
translated into the document language, or vice versa. In addition to direction,
there are multiple methods to perform the translation, four of which we explore
in this paper: word-based, 10-best, context-based, and grammar-based. We build
a feature for each combination of translation direction and method, and train a
model that learns optimal feature weights. On a large forum dataset consisting
of posts in English, Arabic, and Chinese, our novel learn-to-translate approach
was more effective than a strong baseline (p<0.05): translating all text into
English, then training a classifier based only on English (original or
translated) text.

该研究探讨多语种问答中的翻译方法，并提出基于学习的翻译方法，通过模型训练并结合语言分类器实现翻译，结果比强基准模型表现更优。