Virtual assistants are poised to take a dramatic leap forward in terms of
their dialogue capabilities, spurred by recent advances in transformer-based
Large Language Models (LLMs). Yet a major bottleneck to achieving genuinely
transformative task-oriented dialogue capabilities remains the scarcity of high
quality and linguistically sophisticated data. Existing datasets, while
impressive in scale, have limited domain coverage and contain few genuinely
challenging conversational phenomena; those which are present are typically
unlabelled, making it difficult to assess the strengths and weaknesses of
models without time-consuming and costly human evaluation. Moreover, creating
high quality dialogue data has until now required considerable human input,
limiting both the scale of these datasets and the ability to rapidly bootstrap
data for a new target domain. We aim to overcome these issues with LUCID, a
modularised and highly automated LLM-driven data generation system that
produces realistic, diverse and challenging dialogues. We use LUCID to generate
a seed dataset of 4,277 multi-domain, multi-intent conversations across 100
intents to demonstrate its capabilities. The generated conversations include a
wide range of challenging phenomena and diverse user behaviour, conveniently
identifiable via a set of turn-level tags. Finally, we provide separate test
sets for seen and unseen intents, allowing for convenient out-of-distribution
evaluation. We release both the data generation code and the dataset itself.

通过模块化和高度自动化的 LUCID 系统，我们产生了一个包含 4,277 个跨 100 个意图的多领域、多意图对话的种子数据集，其中包含了各种具有挑战性的现象和多样的用户行为。

LUCID: LLM 生成的复杂有趣对话短语

LUCID: LLM-Generated Utterances for Complex and Interesting Dialogues

Most group fairness notions detect unethical biases by computing statistical
parity metrics on a model's output. However, this approach suffers from several
shortcomings, such as philosophical disagreement, mutual incompatibility, and
lack of interpretability. These shortcomings have spurred the research on
complementary bias detection methods that offer additional transparency into
the sources of discrimination and are agnostic towards an a priori decision on
the definition of fairness and choice of protected features. A recent proposal
in this direction is LUCID (Locating Unfairness through Canonical Inverse
Design), where canonical sets are generated by performing gradient descent on
the input space, revealing a model's desired input given a preferred output.
This information about the model's mechanisms, i.e., which feature values are
essential to obtain specific outputs, allows exposing potential unethical
biases in its internal logic. Here, we present LUCID-GAN, which generates
canonical inputs via a conditional generative model instead of gradient-based
inverse design. LUCID-GAN has several benefits, including that it applies to
non-differentiable models, ensures that canonical sets consist of realistic
inputs, and allows to assess proxy and intersectional discrimination. We
empirically evaluate LUCID-GAN on the UCI Adult and COMPAS data sets and show
that it allows for detecting unethical biases in black-box models without
requiring access to the training data.

通过生成规范输入来定位非公平（unfair）行为，LUCID-GAN 是一种新的检测黑盒模型非道德偏见的方法，无需访问训练数据。