Large language models (LLMs) are widely deployed in various downstream tasks,
e.g., auto-completion, aided writing, or chat-based text generation. However,
the considered output candidates of the underlying search algorithm are
under-explored and under-explained. We tackle this shortcoming by proposing a
tree-in-the-loop approach, where a visual representation of the beam search
tree is the central component for analyzing, explaining, and adapting the
generated outputs. To support these tasks, we present generAItor, a visual
analytics technique, augmenting the central beam search tree with various
task-specific widgets, providing targeted visualizations and interaction
possibilities. Our approach allows interactions on multiple levels and offers
an iterative pipeline that encompasses generating, exploring, and comparing
output candidates, as well as fine-tuning the model based on adapted data. Our
case study shows that our tool generates new insights in gender bias analysis
beyond state-of-the-art template-based methods. Additionally, we demonstrate
the applicability of our approach in a qualitative user study. Finally, we
quantitatively evaluate the adaptability of the model to few samples, as
occurring in text-generation use cases.

通过提出一种基于树的循环方法，将生成的输出结果进行分析、解释和适应，我们提出了一个名为 generAItor 的可视化分析技术，它通过在中央的搜索树上使用各种特定任务的小部件来提供目标可视化和交互。我们的技术支持多层次的交互，提供生成、探索、比较输出候选项以及根据适应数据来微调模型的迭代性流程。通过案例研究，我们证明了该工具在性别偏见分析方面产生了超越最先进模板方法的新见解。此外，我们还展示了该方法在定性用户研究中的适用性。最后，我们通过定量评估模型对少量样本的适应能力来评估其在文本生成用例中的可用性。

generAItor：用于语言模型解释性和适应性的树形循环文本生成器

generAItor: Tree-in-the-Loop Text Generation for Language Model  Explainability and Adaptation

Large Language Models (LLMs) have gathered significant attention due to their
impressive performance on a variety of tasks. ChatGPT, developed by OpenAI, is
a recent addition to the family of language models and is being called a
disruptive technology by a few, owing to its human-like text-generation
capabilities. Although, many anecdotal examples across the internet have
evaluated ChatGPT's strength and weakness, only a few systematic research
studies exist. To contribute to the body of literature of systematic research
on ChatGPT, we evaluate the performance of ChatGPT on Abstractive Summarization
by the means of automated metrics and blinded human reviewers. We also build
automatic text classifiers to detect ChatGPT generated summaries. We found that
while text classification algorithms can distinguish between real and generated
summaries, humans are unable to distinguish between real summaries and those
produced by ChatGPT.

本研究通过使用自动度量和盲审人员评估 ChatGPT 在编写摘要上的表现，同时构建自动文本分类器以检测 ChatGPT 生成的摘要，并发现虽然文本分类算法可以区分真实和生成的摘要，但人类无法区分真实摘要和 ChatGPT 生成的摘要。

通过盲评审和文本分类算法比较 ChatGPT 生成的抽象概括和真实概括

Comparing Abstractive Summaries Generated by ChatGPT to Real Summaries  Through Blinded Reviewers and Text Classification Algorithms

Generative adversarial networks (GANs) have achieved significant success in
generating real-valued data. However, the discrete nature of text hinders the
application of GAN to text-generation tasks. Instead of using the standard GAN
objective, we propose to improve text-generation GAN via a novel approach
inspired by optimal transport. Specifically, we consider matching the latent
feature distributions of real and synthetic sentences using a novel metric,
termed the feature-mover's distance (FMD). This formulation leads to a highly
discriminative critic and easy-to-optimize objective, overcoming the
mode-collapsing and brittle-training problems in existing methods. Extensive
experiments are conducted on a variety of tasks to evaluate the proposed model
empirically, including unconditional text generation, style transfer from
non-parallel text, and unsupervised cipher cracking. The proposed model yields
superior performance, demonstrating wide applicability and effectiveness.

通过使用最优传输的方法改进文本生成 GAN，采用新型度量 Feature-Mover's Distance 匹配真实和合成句子的潜在分布，有效解决了现有方法中的模式崩溃和脆弱训练等问题，在无条件文本生成、非平行文本风格转移和无监督密码破解等任务上均取得了优异表现。