We conducted a detailed analysis on the quality of web-mined corpora for two
low-resource languages (making three language pairs, English-Sinhala,
English-Tamil and Sinhala-Tamil). We ranked each corpus according to a
similarity measure and carried out an intrinsic and extrinsic evaluation on
different portions of this ranked corpus. We show that there are significant
quality differences between different portions of web-mined corpora and that
the quality varies across languages and datasets. We also show that, for some
web-mined datasets, Neural Machine Translation (NMT) models trained with their
highest-ranked 25k portion can be on par with human-curated datasets.

我们对两种低资源语言（英语 - 僧伽罗语、英语 - 泰米尔语和僧伽罗语 - 泰米尔语）的网络挖掘语料库的质量进行了详细分析。我们根据相似度度量对每个语料库进行了排序，并对排名的语料库的不同部分进行了内在和外在评估。我们表明，不同部分的网络挖掘语料库之间存在显著的质量差异，而且质量在不同语言和数据集之间存在变化。我们还表明，对于某些网络挖掘数据集，使用最高排名的 25k 部分训练的神经机器翻译（NMT）模型可以与人工策划的数据集媲美。

质量确实重要：对网络挖掘平行语料库的质量和实用性的详细研究

Quality Does Matter: A Detailed Look at the Quality and Utility of  Web-Mined Parallel Corpora

Over the past decade, Artificial Intelligence (AI) has had great success
recently and is being used in a wide range of academic and industrial fields.
More recently, LLMs have made rapid advancements that have propelled AI to a
new level, enabling even more diverse applications and industrial domains with
intelligence, particularly in areas like software engineering and natural
language processing. Nevertheless, a number of emerging trustworthiness
concerns and issues exhibited in LLMs have already recently received much
attention, without properly solving which the widespread adoption of LLMs could
be greatly hindered in practice. The distinctive characteristics of LLMs, such
as the self-attention mechanism, extremely large model scale, and
autoregressive generation schema, differ from classic AI software based on CNNs
and RNNs and present new challenges for quality analysis. Up to the present, it
still lacks universal and systematic analysis techniques for LLMs despite the
urgent industrial demand. Towards bridging this gap, we initiate an early
exploratory study and propose a universal analysis framework for LLMs, LUNA,
designed to be general and extensible, to enable versatile analysis of LLMs
from multiple quality perspectives in a human-interpretable manner. In
particular, we first leverage the data from desired trustworthiness
perspectives to construct an abstract model as an auxiliary analysis asset,
which is empowered by various abstract model construction methods. To assess
the quality of the abstract model, we collect and define a number of evaluation
metrics, aiming at both abstract model level and the semantics level. Then, the
semantics, which is the degree of satisfaction of the LLM w.r.t. the
trustworthiness perspective, is bound to and enriches the abstract model with
semantics, which enables more detailed analysis applications for diverse
purposes.

近年来，人工智能（AI）取得了巨大的成功，并广泛应用于学术和工业领域。然而，长文本语言模型（LLMs）展示出的可信度问题已经引起了广泛关注，为了解决这个问题，我们提出了一个通用的分析框架，LUNA，旨在通过多个质量视角以一种人类可解释的方式对 LLMs 进行分析。

LUNA：大型语言模型的基于模型的通用分析框架

LUNA: A Model-Based Universal Analysis Framework for Large Language  Models

Access to individual-level health data is essential for gaining new insights
and advancing science. In particular, modern methods based on artificial
intelligence rely on the availability of and access to large datasets. In the
health sector, access to individual-level data is often challenging due to
privacy concerns. A promising alternative is the generation of fully synthetic
data, i.e. data generated through a randomised process that have similar
statistical properties as the original data, but do not have a one-to-one
correspondence with the original individual-level records. In this study, we
use a state-of-the-art synthetic data generation method and perform in-depth
quality analyses of the generated data for a specific use case in the field of
nutrition. We demonstrate the need for careful analyses of synthetic data that
go beyond descriptive statistics and provide valuable insights into how to
realise the full potential of synthetic datasets. By extending the methods, but
also by thoroughly analysing the effects of sampling from a trained model, we
are able to largely reproduce significant real-world analysis results in the
chosen use case.

本研究使用最先进的合成数据生成方法并对在营养学领域的特定用例中生成的数据进行深入的质量分析，展示了对合成数据进行仔细分析的必要性，并通过扩展方法，全面分析从训练模型中抽样的效果，从而在所选择的用例中基本复现了重要的真实世界分析结果。