Large language models (LLMs) are capable of generating multiple responses to
a single prompt, yet little effort has been expended to help end-users or
system designers make use of this capability. In this paper, we explore how to
present many LLM responses at once. We design five features, which include both
pre-existing and novel methods for computing similarities and differences
across textual documents, as well as how to render their outputs. We report on
a controlled user study (n=24) and eight case studies evaluating these features
and how they support users in different tasks. We find that the features
support a wide variety of sensemaking tasks and even make tasks previously
considered to be too difficult by our participants now tractable. Finally, we
present design guidelines to inform future explorations of new LLM interfaces.

研究了如何同时呈现多个大型语言模型的响应，设计了包括计算文本文档之间相似性和差异性的预存和新方法以及如何渲染它们的输出的五个特征，通过对用户研究和八个案例研究的评估发现这些特征支持多样的意义理解任务，并使之前被认为过于困难的任务现在变得可行，最后提出了设计指南以指导未来对新型大型语言模型界面的探索。

大规模支持大型语言模型输出的意义构建

Supporting Sensemaking of Large Language Model Outputs at Scale

Current perceptual similarity metrics operate at the level of pixels and
patches. These metrics compare images in terms of their low-level colors and
textures, but fail to capture mid-level similarities and differences in image
layout, object pose, and semantic content. In this paper, we develop a
perceptual metric that assesses images holistically. Our first step is to
collect a new dataset of human similarity judgments over image pairs that are
alike in diverse ways. Critical to this dataset is that judgments are nearly
automatic and shared by all observers. To achieve this we use recent
text-to-image models to create synthetic pairs that are perturbed along various
dimensions. We observe that popular perceptual metrics fall short of explaining
our new data, and we introduce a new metric, DreamSim, tuned to better align
with human perception. We analyze how our metric is affected by different
visual attributes, and find that it focuses heavily on foreground objects and
semantic content while also being sensitive to color and layout. Notably,
despite being trained on synthetic data, our metric generalizes to real images,
giving strong results on retrieval and reconstruction tasks. Furthermore, our
metric outperforms both prior learned metrics and recent large vision models on
these tasks.

本文提出了一个全新的知觉度量标准 DreamSim，通过人类相似度判断数据集的研究得出，该标准可以全面评估图片的相似性，更关注于前景物体和语义内容及对颜色和布局敏感，并且具有很好的一般性能。