We introduce the new task of generating Illustrated Instructions, i.e.,
visual instructions customized to a user's needs. We identify desiderata unique
to this task, and formalize it through a suite of automatic and human
evaluation metrics, designed to measure the validity, consistency, and efficacy
of the generations. We combine the power of large language models (LLMs)
together with strong text-to-image generation diffusion models to propose a
simple approach called StackedDiffusion, which generates such illustrated
instructions given text as input. The resulting model strongly outperforms
baseline approaches and state-of-the-art multimodal LLMs; and in 30% of cases,
users even prefer it to human-generated articles. Most notably, it enables
various new and exciting applications far beyond what static articles on the
web can provide, such as personalized instructions complete with intermediate
steps and pictures in response to a user's individual situation.

生成用户定制的视觉说明书的新任务，通过结合大型语言模型和文本到图像生成扩散模型实现了一种简单方法（StackedDiffusion），在有效性、一致性和效果的度量上远远超过基线方法和最先进的多模态语言模型。

生成图解指引

Generating Illustrated Instructions

Research in natural language processing has demonstrated that the quality of
generations from trained autoregressive language models is significantly
influenced by the used sampling strategy. In this study, we investigate the
impact of different sampling techniques on musical qualities such as diversity
and structure. To accomplish this, we train a high-capacity transformer model
on a vast collection of highly-structured Irish folk melodies and analyze the
musical qualities of the samples generated using distribution truncation
sampling techniques. Specifically, we use nucleus sampling, the recently
proposed "typical sampling", and conventional ancestral sampling. We evaluate
the effect of these sampling strategies in two scenarios: optimal circumstances
with a well-calibrated model and suboptimal circumstances where we
systematically degrade the model's performance. We assess the generated samples
using objective and subjective evaluations. We discover that probability
truncation techniques may restrict diversity and structural patterns in optimal
circumstances, but may also produce more musical samples in suboptimal
circumstances.

本研究探讨了不同的采样技术对音乐质量的影响，训练了一个高容量的变形器模型，使用概率截断采样技术分析样本的音乐特性，在优化和非优化的情况下评估了生成的样本。