We introduce the novel Diffusion Visual Programmer (DVP), a neuro-symbolic
image translation framework. Our proposed DVP seamlessly embeds a
condition-flexible diffusion model within the GPT architecture, orchestrating a
coherent sequence of visual programs (i.e., computer vision models) for various
pro-symbolic steps, which span RoI identification, style transfer, and position
manipulation, facilitating transparent and controllable image translation
processes. Extensive experiments demonstrate DVP's remarkable performance,
surpassing concurrent arts. This success can be attributed to several key
features of DVP: First, DVP achieves condition-flexible translation via
instance normalization, enabling the model to eliminate sensitivity caused by
the manual guidance and optimally focus on textual descriptions for
high-quality content generation. Second, the framework enhances in-context
reasoning by deciphering intricate high-dimensional concepts in feature spaces
into more accessible low-dimensional symbols (e.g., [Prompt], [RoI object]),
allowing for localized, context-free editing while maintaining overall
coherence. Last but not least, DVP improves systemic controllability and
explainability by offering explicit symbolic representations at each
programming stage, empowering users to intuitively interpret and modify
results. Our research marks a substantial step towards harmonizing artificial
image translation processes with cognitive intelligence, promising broader
applications.

我们引入了新颖的扩散可视化编程器 (DVP)，这是一个神经符号图像转换框架。我们提出的 DVP 在 GPT 架构中无缝嵌入了一个条件灵活的扩散模型，为各种符号步骤（包括 RoI 识别、风格转换和位置操作）提供一致的视觉程序序列，实现透明和可控的图像转换过程。大量实验证明了 DVP 的卓越性能，超过了当前艺术的表现。其中 DVP 的几个关键功能是：通过实例标准化实现条件灵活的翻译，使模型消除了手动引导导致的敏感性，并始终专注于用于高质量内容生成的文本描述；框架通过将特征空间中的复杂高维概念解读为更易理解的低维符号（如 [Prompt]、[RoI object]）来增强上下文推理能力，从而实现了局部的、无上下文的编辑和整体的一致性维护；最后，DVP 在每个编程阶段提供明确的符号表示，提高了系统的可控性和可解释性，使用户能够直观地解释和修改结果。我们的研究在将人工图像翻译过程与认知智能相协调方面迈出了重要的一步，具有广泛的应用前景。