The success of models operating on tokenized data has led to an increased
demand for effective tokenization methods, particularly when applied to vision
or auditory tasks, which inherently involve non-discrete data. One of the most
popular tokenization methods is Vector Quantization (VQ), a key component of
several recent state-of-the-art methods across various domains. Typically, a VQ
Variational Autoencoder (VQVAE) is trained to transform data to and from its
tokenized representation. However, since the VQVAE is trained with a
reconstruction objective, there is no constraint for the embeddings to be well
disentangled, a crucial aspect for using them in discriminative tasks.
Recently, several works have demonstrated the benefits of utilizing hyperbolic
spaces for representation learning. Hyperbolic spaces induce compact latent
representations due to their exponential volume growth and inherent ability to
model hierarchical and structured data. In this work, we explore the use of
hyperbolic spaces for vector quantization (HyperVQ), formulating the VQ
operation as a hyperbolic Multinomial Logistic Regression (MLR) problem, in
contrast to the Euclidean K-Means clustering used in VQVAE. Through extensive
experiments, we demonstrate that hyperVQ performs comparably in reconstruction
and generative tasks while outperforming VQ in discriminative tasks and
learning a highly disentangled latent space.

利用超 bolic 空间进行向量量化 (HyperVQ)，通过对比 VQVAE 中的欧几里得 K-Means 聚类，形成了超 bolic 多项式逻辑回归问题的 VQ 操作。通过广泛的实验，我们证明 hyperVQ 在重建和生成任务中的性能相当，而在判别任务中胜过 VQ，并学习高度独立的潜在空间。

HyperVQ: 基于 MLR 的双曲空间向量量化

HyperVQ: MLR-based Vector Quantization in Hyperbolic Space

Image colorization has been attracting the research interests of the
community for decades. However, existing methods still struggle to provide
satisfactory colorized results given grayscale images due to a lack of
human-like global understanding of colors. Recently, large-scale Text-to-Image
(T2I) models have been exploited to transfer the semantic information from the
text prompts to the image domain, where text provides a global control for
semantic objects in the image. In this work, we introduce a colorization model
piggybacking on the existing powerful T2I diffusion model. Our key idea is to
exploit the color prior knowledge in the pre-trained T2I diffusion model for
realistic and diverse colorization. A diffusion guider is designed to
incorporate the pre-trained weights of the latent diffusion model to output a
latent color prior that conforms to the visual semantics of the grayscale
input. A lightness-aware VQVAE will then generate the colorized result with
pixel-perfect alignment to the given grayscale image. Our model can also
achieve conditional colorization with additional inputs (e.g. user hints and
texts). Extensive experiments show that our method achieves state-of-the-art
performance in terms of perceptual quality.

本文介绍了一个基于 T2I 模型的彩色转换模型，利用预训练的 T2I 模型的颜色先验知识，结合扩散引导器产生符合灰度图像视觉语义的潜在色调，接着使用一个灵敏度感知的 VQVAE 模型，生成与给定灰度输出像素完美对齐的彩色结果。在大量的实验中，该模型在感知质量上取得了最先进的性能。