The task of accurate and efficient language translation is an extremely
important information processing task. Machine learning enabled and automated
translation that is accurate and fast is often a large topic of interest in the
machine learning and data science communities. In this study, we examine using
local Generative Pretrained Transformer (GPT) models to perform automated zero
shot black-box, sentence wise, multi-natural-language translation into English
text. We benchmark 16 different open-source GPT models, with no custom
fine-tuning, from the Huggingface LLM repository for translating 50 different
non-English languages into English using translated TED Talk transcripts as the
reference dataset. These GPT model inference calls are performed strictly
locally, on single A100 Nvidia GPUs. Benchmark metrics that are reported are
language translation accuracy, using BLEU, GLEU, METEOR, and chrF text overlap
measures, and wall-clock time for each sentence translation. The best overall
performing GPT model for translating into English text for the BLEU metric is
ReMM-v2-L2-13B with a mean score across all tested languages of $0.152$, for
the GLEU metric is ReMM-v2-L2-13B with a mean score across all tested languages
of $0.256$, for the chrF metric is Llama2-chat-AYT-13B with a mean score across
all tested languages of $0.448$, and for the METEOR metric is ReMM-v2-L2-13B
with a mean score across all tested languages of $0.438$.

使用本地生成预训练转换器（GPT）模型进行零 - shot 黑盒多自然语言翻译成英文文本，评估并比较不同开源 GPT 模型在语言翻译准确性上的表现。

使用生成预训练转换器进行多语言自动翻译到英语的机器翻译

Automated Multi-Language to English Machine Translation Using Generative  Pre-Trained Transformers

Recently there has been a growing concern about machine bias, where trained
statistical models grow to reflect controversial societal asymmetries, such as
gender or racial bias. A significant number of AI tools have recently been
suggested to be harmfully biased towards some minority, with reports of racist
criminal behavior predictors, Iphone X failing to differentiate between two
Asian people and Google photos' mistakenly classifying black people as
gorillas. Although a systematic study of such biases can be difficult, we
believe that automated translation tools can be exploited through gender
neutral languages to yield a window into the phenomenon of gender bias in AI.
In this paper, we start with a comprehensive list of job positions from the
U.S. Bureau of Labor Statistics (BLS) and used it to build sentences in
constructions like "He/She is an Engineer" in 12 different gender neutral
languages such as Hungarian, Chinese, Yoruba, and several others. We translate
these sentences into English using the Google Translate API, and collect
statistics about the frequency of female, male and gender-neutral pronouns in
the translated output. We show that GT exhibits a strong tendency towards male
defaults, in particular for fields linked to unbalanced gender distribution
such as STEM jobs. We ran these statistics against BLS' data for the frequency
of female participation in each job position, showing that GT fails to
reproduce a real-world distribution of female workers. We provide experimental
evidence that even if one does not expect in principle a 50:50 pronominal
gender distribution, GT yields male defaults much more frequently than what
would be expected from demographic data alone.
We are hopeful that this work will ignite a debate about the need to augment
current statistical translation tools with debiasing techniques which can
already be found in the scientific literature.

本文研究了机器偏见和性别偏见，使用性别中性语言对 Google 翻译进行了实验，结果显示翻译的默认性别更偏向于男性，特别是在 STEM 领域中，这种偏见程度远超实际分布，因此需要对当前的统计翻译工具进行偏误纠正。