Creative coding tasks are often exploratory in nature. When producing digital
artwork, artists usually begin with a high-level semantic construct such as a
"stained glass filter" and programmatically implement it by varying code
parameters such as shape, color, lines, and opacity to produce visually
appealing results. Based on interviews with artists, it can be effortful to
translate semantic constructs to program syntax, and current programming tools
don't lend well to rapid creative exploration. To address these challenges, we
introduce Spellburst, a large language model (LLM) powered creative-coding
environment. Spellburst provides (1) a node-based interface that allows artists
to create generative art and explore variations through branching and merging
operations, (2) expressive prompt-based interactions to engage in semantic
programming, and (3) dynamic prompt-driven interfaces and direct code editing
to seamlessly switch between semantic and syntactic exploration. Our evaluation
with artists demonstrates Spellburst's potential to enhance creative coding
practices and inform the design of computational creativity tools that bridge
semantic and syntactic spaces.

通过 Spellburst，我们引入了一个大型语言模型（LLM）驱动的创意编码环境，提供了节点式界面、表达式提示交互以及动态提示驱动界面和直接代码编辑等功能，以弥合语义和语法领域之间的鸿沟，并增强创意编码实践。

Spellburst: 自然语言提示的基于节点的探索性创意编码接口

Spellburst: A Node-based Interface for Exploratory Creative Coding with  Natural Language Prompts

This book is the result of a seminar in which we reviewed multimodal
approaches and attempted to create a solid overview of the field, starting with
the current state-of-the-art approaches in the two subfields of Deep Learning
individually. Further, modeling frameworks are discussed where one modality is
transformed into the other, as well as models in which one modality is utilized
to enhance representation learning for the other. To conclude the second part,
architectures with a focus on handling both modalities simultaneously are
introduced. Finally, we also cover other modalities as well as general-purpose
multi-modal models, which are able to handle different tasks on different
modalities within one unified architecture. One interesting application
(Generative Art) eventually caps off this booklet.

这篇论文概述了多模态方法中的最新技术和框架，并重点介绍了 Deep Learning、模型框架、特征表示学习以及生成艺术等关键点。

多模态深度学习

Multimodal Deep Learning

This paper argues that generative art driven by conformance to a visual
and/or semantic corpus lacks the necessary criteria to be considered creative.
Among several issues identified in the literature, we focus on the fact that
generative adversarial networks (GANs) that create a single image, in a vacuum,
lack a concept of novelty regarding how their product differs from previously
created ones. We envision that an algorithm that combines the novelty
preservation mechanisms in evolutionary algorithms with the power of GANs can
deliberately guide its creative process towards output that is both good and
novel. In this paper, we use recent advances in image generation based on
semantic prompts using OpenAI's CLIP model, interrupting the GAN's iterative
process with short cycles of evolutionary divergent search. The results of
evolution are then used to continue the GAN's iterative process; we hypothesise
that this intervention will lead to more novel outputs. Testing our hypothesis
using novelty search with local competition, a quality-diversity evolutionary
algorithm that can increase visual diversity while maintaining quality in the
form of adherence to the semantic prompt, we explore how different notions of
visual diversity can affect both the process and the product of the algorithm.
Results show that even a simplistic measure of visual diversity can help
counter a drift towards similar images caused by the GAN. This first experiment
opens a new direction for introducing higher intentionality and a more nuanced
drive for GANs.

通过结合进化算法的新颖性保持机制和生成对抗网络的优势，我们利用 OpenAI 的 CLIP 模型与短周期的进化分散搜索来指导 GAN 的创造过程，以此来创造出更具有新颖性和优秀性的艺术作品。

将多样性引入 AI 艺术

Seeding Diversity into AI Art

This paper presents a probabilistic extension of the well-known cellular
automaton, Game of Life. In Game of Life, cells are placed in a grid and then
watched as they evolve throughout subsequent generations, as dictated by the
rules of the game. In our extension, called ProbLife, these rules now have
probabilities associated with them. Instead of cells being either dead or
alive, they are denoted by their chance to live. After presenting the rules of
ProbLife and its underlying characteristics, we show a concrete implementation
in ProbLog, a probabilistic logic programming system. We use this to generate
different images, as a form of rule-based generative art.

本文展示了 Game of Life 的概率扩展 ProbLife，使用与其关联的 ProbLog 构建了基于规则的生成艺术，探讨了概率元胞自动机的特征和规则。