We introduce RealmDreamer, a technique for generation of general
forward-facing 3D scenes from text descriptions. Our technique optimizes a 3D
Gaussian Splatting representation to match complex text prompts. We initialize
these splats by utilizing the state-of-the-art text-to-image generators,
lifting their samples into 3D, and computing the occlusion volume. We then
optimize this representation across multiple views as a 3D inpainting task with
image-conditional diffusion models. To learn correct geometric structure, we
incorporate a depth diffusion model by conditioning on the samples from the
inpainting model, giving rich geometric structure. Finally, we finetune the
model using sharpened samples from image generators. Notably, our technique
does not require video or multi-view data and can synthesize a variety of
high-quality 3D scenes in different styles, consisting of multiple objects. Its
generality additionally allows 3D synthesis from a single image.

介绍了一种名为 RealmDreamer 的技术，可以通过文本描述生成通用的正面视图 3D 场景，并且具备高质量和多样性。该技术利用先进的文本到图像生成器初始化 3D 平面，并通过多个视图进行 3D 修补和基于图像的扩散建模，从而生成富含几何结构的场景。最后，通过图像生成器的锐化样本对模型进行微调，而无需视频或多视角数据，同时还可以从单个图像合成 3D 场景。

RealmDreamer: 基于文本驱动的带修补和深度扩散的 3D 场景生成

RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth  Diffusion

Artificial Intelligence has outperformed human experts in functional tasks
such as chess and baduk. How about creative tasks? This paper evaluates AI's
capability in the creative domain compared to human experts, which little
research has been conducted so far. We propose a novel Prompt-for-Prompt to
generate social media creatives via prompt augmentation by Large Language
Models. We take the most popular Instagram posts (with the biggest number of
like clicks) in top brands' Instagram accounts to create social media
creatives. We give GPT 4 several prompt instructions with text descriptions to
generate the most effective prompts for cutting-edge text-to-image generators:
Midjourney, DALL E 3, and Stable Diffusion. LLM-augmented prompts can boost
AI's abilities by adding objectives, engagement strategy, lighting and brand
consistency for social media image creation. We conduct an extensive human
evaluation experiment, and find that AI excels human experts, and Midjourney is
better than the other text-to-image generators. Surprisingly, unlike
conventional wisdom in the social media industry, prompt instruction including
eye-catching shows much poorer performance than those including natural.
Regarding the type of creatives, AI improves creatives with animals or products
but less with real people. Also, AI improves creatives with short text
descriptions more than with long text descriptions, because there is more room
for AI to augment prompts with shorter descriptions.

通过大型语言模型利用 Prompt-for-Prompt 生成社交媒体创意，评估人工智能在创造型任务中的能力，超过人类专家并提高社交媒体图像创作的效果。

AI 能在社交媒体创意制作方面超越人类专家吗？

Can AI Outperform Human Experts in Creating Social Media Creatives?

Generative artificial intelligence (AI) refers to algorithms that create
synthetic but realistic output. Diffusion models currently offer state of the
art performance in generative AI for images. They also form a key component in
more general tools, including text-to-image generators and large language
models. Diffusion models work by adding noise to the available training data
and then learning how to reverse the process. The reverse operation may then be
applied to new random data in order to produce new outputs. We provide a brief
introduction to diffusion models for applied mathematicians and statisticians.
Our key aims are (a) to present illustrative computational examples, (b) to
give a careful derivation of the underlying mathematical formulas involved, and
(c) to draw a connection with partial differential equation (PDE) diffusion
models. We provide code for the computational experiments. We hope that this
topic will be of interest to advanced undergraduate students and postgraduate
students. Portions of the material may also provide useful motivational
examples for those who teach courses in stochastic processes, inference,
machine learning, PDEs or scientific computing.

生成人工智能是指生成合成但逼真的输出的算法。扩散模型在图像生成人工智能中目前达到最高水平，并在包括文本到图像生成器和大型语言模型在内的更通用工具中起着关键作用。

面向应用数学家的生成人工智能扩散模型简介

Diffusion Models for Generative Artificial Intelligence: An Introduction  for Applied Mathematicians

Recent advances in text-to-image generators have led to substantial
capabilities in image generation. However, the complexity of prompts acts as a
bottleneck in the quality of images generated. A particular under-explored
facet is the ability of generative models to create high-quality images
comprising multiple components given as a prior. In this paper, we propose and
validate a metric called Components Inclusion Score (CIS) to evaluate the
extent to which a model can correctly generate multiple components. Our results
reveal that the evaluated models struggle to incorporate all the visual
elements from prompts with multiple components (8.53% drop in CIS per component
for all evaluated models). We also identify a significant decline in the
quality of the images and context awareness within an image as the number of
components increased (15.91% decrease in inception Score and 9.62% increase in
Frechet Inception Distance). To remedy this issue, we fine-tuned Stable
Diffusion V2 on a custom-created test dataset with multiple components,
outperforming its vanilla counterpart. To conclude, these findings reveal a
critical limitation in existing text-to-image generators, shedding light on the
challenge of generating multiple components within a single image using a
complex prompt.

本研究旨在提出并验证一种名为 “组件包含分数（CIS）” 的评估指标，评估模型能够正确生成多个组件的程度，并发现现有文本到图像生成器在使用复杂提示时生成多个组件的能力存在关键限制。