Large language models that exhibit instruction-following behaviour represent
one of the biggest recent upheavals in conversational interfaces, a trend in
large part fuelled by the release of OpenAI's ChatGPT, a proprietary large
language model for text generation fine-tuned through reinforcement learning
from human feedback (LLM+RLHF). We review the risks of relying on proprietary
software and survey the first crop of open-source projects of comparable
architecture and functionality. The main contribution of this paper is to show
that openness is differentiated, and to offer scientific documentation of
degrees of openness in this fast-moving field. We evaluate projects in terms of
openness of code, training data, model weights, RLHF data, licensing,
scientific documentation, and access methods. We find that while there is a
fast-growing list of projects billing themselves as 'open source', many inherit
undocumented data of dubious legality, few share the all-important
instruction-tuning (a key site where human annotation labour is involved), and
careful scientific documentation is exceedingly rare. Degrees of openness are
relevant to fairness and accountability at all points, from data collection and
curation to model architecture, and from training and fine-tuning to release
and deployment.

本文简要回顾了 OpenAI 的 ChatGPT 及其开源类似项目，并评估它们在代码可访问性、数据、许可证、训练以及 fine-tuning 等方面的开放程度。我们发现，尽管有许多声称为 “开源” 的项目，但是其中大部分存在不确定的数据来源以及缺乏充分的文档说明，缺少重要的注释调整，有意识的科学文档非常罕见，而这些因素在公平性和责任性方面都显得非常重要。

揭开 ChatGPT：跟踪指导调整的文本生成器 中的开放性、透明度和问责制

Opening up ChatGPT: Tracking openness, transparency, and accountability  in instruction-tuned text generators

Many powerful computing technologies rely on implicit and explicit data
contributions from the public. This dependency suggests a potential source of
leverage for the public in its relationship with technology companies: by
reducing, stopping, redirecting, or otherwise manipulating data contributions,
the public can reduce the effectiveness of many lucrative technologies. In this
paper, we synthesize emerging research that seeks to better understand and help
people action this \textit{data leverage}. Drawing on prior work in areas
including machine learning, human-computer interaction, and fairness and
accountability in computing, we present a framework for understanding data
leverage that highlights new opportunities to change technology company
behavior related to privacy, economic inequality, content moderation and other
areas of societal concern. Our framework also points towards ways that
policymakers can bolster data leverage as a means of changing the balance of
power between the public and tech companies.

本文综合了机器学习、人机交互和公平与责任等领域的相关研究，提出了一个以数据杠杆为核心的框架，探讨了杠杆对于改变科技公司与公众体系中的隐私、经济不平等、内容审核等方面的影响，并展示了政策制定者如何促进数据杠杆以改变公众与科技公司之间的力量平衡。

数据利用：赋权公众与科技企业的关系框架

Data Leverage: A Framework for Empowering the Public in its Relationship  with Technology Companies

Attention mechanisms are ubiquitous components in neural architectures
applied to natural language processing. In addition to yielding gains in
predictive accuracy, attention weights are often claimed to confer
interpretability, purportedly useful both for providing insights to
practitioners and for explaining why a model makes its decisions to
stakeholders. We call the latter use of attention mechanisms into question by
demonstrating a simple method for training models to produce deceptive
attention masks. Our method diminishes the total weight assigned to designated
impermissible tokens, even when the models can be shown to nevertheless rely on
these features to drive predictions. Across multiple models and tasks, our
approach manipulates attention weights while paying surprisingly little cost in
accuracy. Through a human study, we show that our manipulated attention-based
explanations deceive people into thinking that predictions from a model biased
against gender minorities do not rely on the gender. Consequently, our results
cast doubt on attention's reliability as a tool for auditing algorithms in the
context of fairness and accountability.

本研究提出一种简单方法，通过训练模型生成欺骗性的注意力掩码，从而质疑注意机制的可靠性，因为通过操纵注意权重，即使我们可以证明模型仍然依赖这些特征来驱动预测，我们的方法会减少指定的不允许的标记被分配的总权重。通过人类研究，我们展示了我们操作注意力的解释会欺骗人们认为偏见的模型的预测不依赖性别，从而对注意力在公平性和问责制的算法审计工具上的可靠性产生怀疑。