In this paper, we formulate the colorization problem into a multinomial
classification problem and then apply a weighted function to classes. We
propose a set of formulas to transform color values into color classes and vice
versa. To optimize the classes, we experiment with different bin sizes for
color class transformation. Observing class appearance, standard deviation, and
model parameters on various extremely large-scale real-time images in practice
we propose 532 color classes for our classification task. During training, we
propose a class-weighted function based on true class appearance in each batch
to ensure proper saturation of individual objects. We adjust the weights of the
major classes, which are more frequently observed, by lowering them, while
escalating the weights of the minor classes, which are less commonly observed.
In our class re-weight formula, we propose a hyper-parameter for finding the
optimal trade-off between the major and minor appeared classes. As we apply
regularization to enhance the stability of the minor class, occasional minor
noise may appear at the object's edges. We propose a novel object-selective
color harmonization method empowered by the Segment Anything Model (SAM) to
refine and enhance these edges. We propose two new color image evaluation
metrics, the Color Class Activation Ratio (CCAR), and the True Activation Ratio
(TAR), to quantify the richness of color components. We compare our proposed
model with state-of-the-art models using six different dataset: Place, ADE,
Celeba, COCO, Oxford 102 Flower, and ImageNet, in qualitative and quantitative
approaches. The experimental results show that our proposed model outstrips
other models in visualization, CNR and in our proposed CCAR and TAR measurement
criteria while maintaining satisfactory performance in regression (MSE, PSNR),
similarity (SSIM, LPIPS, UIUI), and generative criteria (FID).

本研究提出了一种基于多项式分类问题的上色方法，并应用加权函数于类别，通过将颜色值转换为颜色类别及其逆转换的一系列公式，优化了颜色类别转换的不同 bin 尺寸。在实际应用中，通过观察类别外观、标准差和模型参数，基于大规模实时图像，我们为分类任务提出了 532 种颜色类别。同时，在训练过程中，我们提出了一种基于每批次真实类别外观的类别加权函数，以确保正确饱和个别对象。我们调整了较频繁观察到的主要类别的权重，通过降低它们的权重，同时提高较少常见的次要类别的权重。我们的类别重新权重函数中提出了一个超参数，用于找到主要和次要出现类别之间的最佳折衷。为了增强次要类别的稳定性，我们应用正则化方法，但在对象的边缘可能会出现偶发的次要噪点。因此，我们提出了一种由 Segment Anything Model（SAM）强化的新颖的对象选择性颜色均衡方法，以改进和增强这些边缘。此外，我们还提出了两个新的颜色图像评估指标：色彩类别激活比率（CCAR）和真实激活比率（TAR），以量化颜色分量的丰富性。我们通过在六个不同数据集上进行定性和定量比较，包括 Place，ADE，Celeba，COCO，Oxford 102 Flower 和 ImageNet，将我们提出的模型与现有模型进行了比较。实验结果表明，我们的模型在可视化、CNR 以及我们提出的 CCAR 和 TAR 测量指标方面优于其他模型，同时在回归（MSE，PSNR）、相似度（SSIM，LPIPS，UIUI）和生成性能（FID）方面表现仍是令人满意的。

CCC++：基于分段任意建模（SAM）增强的物体选择性颜色协调的优化彩色分类上色

CCC++: Optimized Color Classified Colorization with Segment Anything  Model (SAM) Empowered Object Selective Color Harmonization

Automatic colorization of gray images with objects of different colors and
sizes is challenging due to inter- and intra-object color variation and the
small area of the main objects due to extensive backgrounds. The learning
process often favors dominant features, resulting in a biased model. In this
paper, we formulate the colorization problem into a multinomial classification
problem and then apply a weighted function to classes. We propose a set of
formulas to transform color values into color classes and vice versa. Class
optimization and balancing feature distribution are the keys for good
performance. Observing class appearance on various extremely large-scale
real-time images in practice, we propose 215 color classes for our colorization
task. During training, we propose a class-weighted function based on true class
appearance in each batch to ensure proper color saturation of individual
objects. We establish a trade-off between major and minor classes to provide
orthodox class prediction by eliminating major classes' dominance over minor
classes. As we apply regularization to enhance the stability of the minor
class, occasional minor noise may appear at the object's edges. We propose a
novel object-selective color harmonization method empowered by the SAM to
refine and enhance these edges. We propose a new color image evaluation metric,
the Chromatic Number Ratio (CNR), to quantify the richness of color components.
We compare our proposed model with state-of-the-art models using five different
datasets: ADE, Celeba, COCO, Oxford 102 Flower, and ImageNet, in both
qualitative and quantitative approaches. The experimental results show that our
proposed model outstrips other models in visualization and CNR measurement
criteria while maintaining satisfactory performance in regression (MSE, PSNR),
similarity (SSIM, LPIPS, UIQI), and generative criteria (FID).

自动着色技术面临物体颜色差异、尺寸变化、背景干扰等挑战。本文通过多项式分类方法解决了这一问题，提出了颜色值与颜色类别之间的转换公式，并通过类别优化和平衡特征分布来提升性能。作者还提出了基于真实类别外观的加权函数，用于训练过程中的类别饱和度调整，并通过抑制主要类别的影响实现了对次要类别的预测。同时，作者还提出了一种基于 SAM 的对象选择性颜色协调方法，用于增强图像边缘。通过与其他模型在多个数据集上的比较实验证明，该模型在可视化和颜色丰富度（CNR）方面优于其他模型，并在回归（MSE，PSNR）、相似度（SSIM，LPIPS，UIQI）和生成能力（FID）等指标上表现令人满意。