This work is an update of a previous paper on the same topic published a few
years ago. With the dramatic progress in generative modeling, a suite of new
quantitative and qualitative techniques to evaluate models has emerged.
Although some measures such as Inception Score, Frechet Inception Distance,
Precision-Recall, and Perceptual Path Length are relatively more popular, GAN
evaluation is not a settled issue and there is still room for improvement.
Here, I describe new dimensions that are becoming important in assessing models
(e.g. bias and fairness) and discuss the connection between GAN evaluation and
deepfakes. These are important areas of concern in the machine learning
community today and progress in GAN evaluation can help mitigate them.

本文讨论了生成模型的进展和评估技术，以及 GAN 评估和深度伪造之间的联系，探讨了在机器学习社区中日益重要的公平和偏见问题，并指出改进 GAN 评估可以有助于减轻这些问题。

GAN 评估指标的优缺点：新进展

Pros and Cons of GAN Evaluation Measures: New Developments

We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities.

对 146 篇分析自然语言处理系统中 “偏见” 的论文进行调查，发现大部分这些论文的动机模糊、不一致且缺少规范性推理，同时发现这些论文提出的量化技术与动机不符，并未接触自然语言以外的相关文献。基于这些发现，为分析自然语言处理系统中的 “偏见” 工作提出了三个建议。这些建议包括更加认识语言与社会等级之间的关系，鼓励研究人员和实践者表达对 “偏见” 的概念化，即什么样的系统行为对谁有害，以及为什么，同时也要关注受自然语言处理系统影响的社区成员的生活经验，质疑和重新构想技术人员与这样的社区之间的权力关系。