This essay proposes and explores the concept of Collage for the design of AI
writing tools, transferred from avant-garde literature with four facets: 1)
fragmenting text in writing interfaces, 2) juxtaposing voices (content vs
command), 3) integrating material from multiple sources (e.g. text
suggestions), and 4) shifting from manual writing to editorial and
compositional decision-making, such as selecting and arranging snippets. The
essay then employs Collage as an analytical lens to analyse the user interface
design of recent AI writing tools, and as a constructive lens to inspire new
design directions. Finally, a critical perspective relates the concerns that
writers historically expressed through literary collage to AI writing tools. In
a broad view, this essay explores how literary concepts can help advance design
theory around AI writing tools. It encourages creators of future writing tools
to engage not only with new technological possibilities, but also with past
writing innovations.

这篇论文提出并探索了以拼贴艺术为设计概念的人工智能写作工具，重点是通过把文本碎片化、声音对立、多源材料整合和选择与排列片段等方式，分析最近的 AI 写作工具的用户界面设计，并通过创造性的眼光激发新的设计方向，同时将历史上作家们通过文学拼贴所表达的关切与 AI 写作工具联系起来，探索了如何借鉴文学概念来推进 AI 写作工具的设计理论，鼓励未来的写作工具创作者不仅要关注新技术的可能性，还要关注过去的写作创新。

拼贴是新的写作形式：探索 AI 工具中的文本和用户界面的碎片化

Collage is the New Writing: Exploring the Fragmentation of Text and User  Interfaces in AI Tools

Large models training is plagued by the intense compute cost and limited
hardware memory. A practical solution is low-precision representation but is
troubled by loss in numerical accuracy and unstable training rendering the
model less useful. We argue that low-precision floating points can perform well
provided the error is properly compensated at the critical locations in the
training process. We propose Collage which utilizes multi-component float
representation in low-precision to accurately perform operations with numerical
errors accounted. To understand the impact of imprecision to training, we
propose a simple and novel metric which tracks the lost information during
training as well as differentiates various precision strategies. Our method
works with commonly used low-precision such as half-precision ($16$-bit
floating points) and can be naturally extended to work with even lower
precision such as $8$-bit. Experimental results show that pre-training using
Collage removes the requirement of using $32$-bit floating-point copies of the
model and attains similar/better training performance compared to $(16,
32)$-bit mixed-precision strategy, with up to $3.7\times$ speedup and $\sim
15\%$ to $23\%$ less memory usage in practice.

低精度浮点数的训练可以通过适当的误差补偿在关键位置以提供良好的性能，Collage 方法使用多组分浮点表示来准确计算带有数值误差的操作，并且通过调用新的度量方法能够追踪训练期间丢失的信息，同时针对不同的精度策略进行区分。实验结果表明，使用 Collage 进行预训练可以去除使用 32 位浮点数副本的要求，并在实践中实现与 (16, 32) 位混合精度策略相似或更好的训练性能，加速了高达 3.7 倍，内存使用量减少了约 15％至 23％。

轻量级低精度策略用于 LLM 训练

Collage: Light-Weight Low-Precision Strategy for LLM Training

Collage is a creative art form that uses diverse material scraps as a base
unit to compose a single image. Although pixel-wise generation techniques can
reproduce a target image in collage style, it is not a suitable method due to
the solid stroke-by-stroke nature of the collage form. While some previous
works for stroke-based rendering produced decent sketches and paintings,
collages have received much less attention in research despite their popularity
as a style. In this paper, we propose a method for learning to make collages
via reinforcement learning without the need for demonstrations or collage
artwork data. We design the collage Markov Decision Process (MDP), which allows
the agent to handle various materials and propose a model-based soft
actor-critic to mitigate the agent's training burden derived from the
sophisticated dynamics of collage. Moreover, we devise additional techniques
such as active material selection and complexity-based multi-scale collage to
handle target images at any size and enhance the results' aesthetics by placing
relatively more scraps in areas of high complexity. Experimental results show
that the trained agent appropriately selected and pasted materials to
regenerate the target image into a collage and obtained a higher evaluation
score on content and style than pixel-wise generation methods. Code is
available at this https URL

通过强化学习训练的方法，本研究提出了一种无需演示或拼贴艺术作品数据即可学习制作拼贴艺术的方法。通过设计了拼贴马尔科夫决策过程（MDP），并使用基于模型的软性演员 - 评论家算法来降低拼贴过程的训练难度；此外，本研究还采用了主动材料选择和基于复杂度的多尺度拼贴等技术来处理任意尺寸的目标图像，并通过在复杂区域放置更多碎片来提高结果的美感。实验结果表明，训练得到的智能体能够适当地选择和粘贴材料，以拼贴的形式重新生成目标图像，并在内容和风格评估方面获得比基于像素生成方法更高的评分。