An indigenous perspective on the effectiveness of debiasing techniques for
pre-trained language models (PLMs) is presented in this paper. The current
techniques used to measure and debias PLMs are skewed towards the US racial
biases and rely on pre-defined bias attributes (e.g. "black" vs "white"). Some
require large datasets and further pre-training. Such techniques are not
designed to capture the underrepresented indigenous populations in other
countries, such as M\=aori in New Zealand. Local knowledge and understanding
must be incorporated to ensure unbiased algorithms, especially when addressing
a resource-restricted society.

本文提供了对已预训练语言模型去偏见技术效果的土著观点。 现有技术主要针对美国种族偏见，无法有效捕捉其他国家（如新西兰的毛利人）中代表性不足的土著人群，需要结合本地知识和理解来确保算法的公正性，特别是解决资源受限的社会问题。

去偏见技巧的有效性：本土定性分析

Effectiveness of Debiasing Techniques: An Indigenous Qualitative  Analysis

State-of-the-art conversational agents have advanced significantly in
conjunction with the use of large transformer-based language models. However,
even with these advancements, conversational agents still lack the ability to
produce responses that are informative and coherent with the local context. In
this work, we propose a dialog framework that incorporates both local knowledge
as well as users' past dialogues to generate high quality conversations. We
introduce an approach to build a dataset based on Reddit conversations, where
outbound URL links are widely available in the conversations and the
hyperlinked documents can be naturally included as local external knowledge.
Using our framework and dataset, we demonstrate that incorporating local
knowledge can largely improve informativeness, coherency and realisticness
measures using human evaluations. In particular, our approach consistently
outperforms the state-of-the-art conversational model on the Reddit dataset
across all three measures. We also find that scaling the size of our models
from 117M to 8.3B parameters yields consistent improvement of validation
perplexity as well as human evaluated metrics. Our model with 8.3B parameters
can generate human-like responses as rated by various human evaluations in a
single-turn dialog setting.

本文介绍了一个基于 Reddit 对话数据集的对话框架，并探讨如何整合局部知识以改善对话质量，研究结果表明，局部知识的整合显著提高了信息性、连贯性和逼真度，且所提出的方法在 Reddit 数据集上表现优于现有方法。同时发现，将模型的参数规模从 117M 扩展到 8.3B 能够显著提高验证困惑度和人工评估指标，使模型在单轮对话设置中生成的响应更加类人。