This paper introduces a novel approach to sketch colourisation, inspired by
the universal childhood activity of colouring and its professional applications
in design and story-boarding. Striking a balance between precision and
convenience, our method utilises region masks and colour palettes to allow
intuitive user control, steering clear of the meticulousness of manual colour
assignments or the limitations of textual prompts. By strategically combining
ControlNet and staged generation, incorporating Stable Diffusion v1.5, and
leveraging BLIP-2 text prompts, our methodology facilitates faithful image
generation and user-directed colourisation. Addressing challenges of local and
global consistency, we employ inventive solutions such as an inversion scheme,
guided sampling, and a self-attention mechanism with a scaling factor. The
resulting tool is not only fast and training-free but also compatible with
consumer-grade Nvidia RTX 4090 Super GPUs, making it a valuable asset for both
creative professionals and enthusiasts in various fields. Project Page:
https://chaitron.github.io/SketchDeco/

这篇论文介绍了一种新颖的素描彩色化方法，利用区域掩码和调色板实现了对图像的直观用户控制，同时结合了控制网和分段生成、稳定扩散 v1.5、BLIP-2 文本提示等创新策略，解决了局部和全局一致性的挑战，生成了忠实的图像彩色化工具。

SketchDeco: 用色彩装饰黑白素描

SketchDeco: Decorating B&W Sketches with Colour

This paper tackles the problem of novel view synthesis from a single image.
In particular, we target real-world scenes with rich geometric structure, a
challenging task due to the large appearance variations of such scenes and the
lack of simple 3D models to represent them. Modern, learning-based approaches
mostly focus on appearance to synthesize novel views and thus tend to generate
predictions that are inconsistent with the underlying scene structure. By
contrast, in this paper, we propose to exploit the 3D geometry of the scene to
synthesize a novel view. Specifically, we approximate a real-world scene by a
fixed number of planes, and learn to predict a set of homographies and their
corresponding region masks to transform the input image into a novel view. To
this end, we develop a new region-aware geometric transform network that
performs these multiple tasks in a common framework. Our results on the outdoor
KITTI and the indoor ScanNet datasets demonstrate the effectiveness of our
network in generating high quality synthetic views that respect the scene
geometry, thus outperforming the state-of-the-art methods.

本文提出了一种利用场景 3D 几何信息进行新视角生成的方法，通过学习区域感知几何转换网络实现输入图像到目标视角的变换，并在 KITTI 和 ScanNet 数据集上取得了优于现有方法的高质量生成效果。