Automatic art analysis employs different image processing techniques to
classify and categorize works of art. When working with artistic images, we
need to take into account further considerations compared to classical image
processing. This is because such artistic paintings change drastically
depending on the author, the scene depicted, and their artistic style. This can
result in features that perform very well in a given task but do not grasp the
whole of the visual and symbolic information contained in a painting. In this
paper, we show how the features obtained from different tasks in artistic image
classification are suitable to solve other ones of similar nature. We present
different methods to improve the generalization capabilities and performance of
artistic classification systems. Furthermore, we propose an explainable
artificial intelligence method to map known visual traits of an image with the
features used by the deep learning model considering fuzzy rules. These rules
show the patterns and variables that are relevant to solve each task and how
effective is each of the patterns found. Our results show that our proposed
context-aware features can achieve up to $6\%$ and $26\%$ more accurate results
than other context- and non-context-aware solutions, respectively, depending on
the specific task. We also show that some of the features used by these models
can be more clearly correlated to visual traits in the original image than
others.

自动艺术分析利用不同的图像处理技术对艺术作品进行分类和归类。本文展示了如何利用艺术图像分类中不同任务获得的特征来解决其他具有相似性质的任务，并提出了改进艺术分类系统的泛化能力和性能的不同方法。此外，我们还提出了一种可解释的人工智能方法，通过模糊规则将图像的已知视觉特征与深度学习模型使用的特征进行映射。我们的结果表明，相较于其他上下文感知和非上下文感知的解决方案，我们提出的上下文感知特征在特定任务上可以获得高达 6% 和 26% 的更准确的结果。我们还展示了其中一些模型使用的特征与原始图像的视觉特征之间的相关性更明显。

ARTxAI：使用模糊技术基于可解释的人工智能为艺术图像进行深度表示学习

ARTxAI: Explainable Artificial Intelligence Curates Deep Representation  Learning for Artistic Images using Fuzzy Techniques

Automatic art analysis aims to classify and retrieve artistic representations
from a collection of images by using computer vision and machine learning
techniques. In this work, we propose to enhance visual representations from
neural networks with contextual artistic information. Whereas visual
representations are able to capture information about the content and the style
of an artwork, our proposed context-aware embeddings additionally encode
relationships between different artistic attributes, such as author, school, or
historical period. We design two different approaches for using context in
automatic art analysis. In the first one, contextual data is obtained through a
multi-task learning model, in which several attributes are trained together to
find visual relationships between elements. In the second approach, context is
obtained through an art-specific knowledge graph, which encodes relationships
between artistic attributes. An exhaustive evaluation of both of our models in
several art analysis problems, such as author identification, type
classification, or cross-modal retrieval, show that performance is improved by
up to 7.3% in art classification and 37.24% in retrieval when context-aware
embeddings are used.

本文提出了两种方法：使用多任务学习模型获取上下文数据以在自动艺术分析中使用上下文，以及通过艺术特定的知识图谱获得上下文，为视觉表示增加上下文艺术信息，以实现艺术作品的分类和检索。

用于自动艺术分析的上下文感知嵌入

Context-Aware Embeddings for Automatic Art Analysis

Automatic art analysis has been mostly focused on classifying artworks into
different artistic styles. However, understanding an artistic representation
involves more complex processes, such as identifying the elements in the scene
or recognizing author influences. We present SemArt, a multi-modal dataset for
semantic art understanding. SemArt is a collection of fine-art painting images
in which each image is associated to a number of attributes and a textual
artistic comment, such as those that appear in art catalogues or museum
collections. To evaluate semantic art understanding, we envisage the Text2Art
challenge, a multi-modal retrieval task where relevant paintings are retrieved
according to an artistic text, and vice versa. We also propose several models
for encoding visual and textual artistic representations into a common semantic
space. Our best approach is able to find the correct image within the top 10
ranked images in the 45.5% of the test samples. Moreover, our models show
remarkable levels of art understanding when compared against human evaluation.

该研究提出了 SemArt，一个多模态数据集，以进行语义艺术理解，包括将视觉和文本表现编码成共同的语义空间，以及 Text2Art 挑战，即通过艺术文本检索相关艺术品。在测试样本中，最佳方法可以在前 10 个排名的图像中找到正确的图像率达 45.5％。