Skin cancer is one of the most prevalent forms of human cancer. It is
recognized mainly visually, beginning with clinical screening and continuing
with the dermoscopic examination, histological assessment, and specimen
collection. Deep convolutional neural networks (CNNs) perform highly segregated
and potentially universal tasks against a classified finegrained object. This
research proposes a novel multi-class prediction framework that classifies skin
lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative
Adversarial Networks) are utilized to tackle the class imbalance. The framework
consists of four main phases: ViTGANs, Image processing, and explainable AI.
Phase 1 consists of generating synthetic images to balance all the classes in
the dataset. Phase 2 consists of applying different data augmentation
techniques and morphological operations to increase the size of the data.
Phases 3 & 4 involve developing a ViT model for edge computing systems that can
identify patterns and categorize skin lesions from the user's skin visible in
the image. In phase 3, after classifying the lesions into the desired class
with ViT, we will use explainable AI (XAI) that leads to more explainable
results (using activation maps, etc.) while ensuring high predictive accuracy.
Real-time images of skin diseases can capture by a doctor or a patient using
the camera of a mobile application to perform an early examination and
determine the cause of the skin lesion. The whole framework is compared with
the existing frameworks for skin lesion detection.

本文提出了一个新的多类预测框架，基于 ViT 和 ViTGAN，在使用可解释 AI 的情况下，对皮肤病变分类。该框架分四个阶段，引入了生成对抗网络来解决类不平衡问题，使用卷积神经网络实现分类，实验结果相较于现有框架有所改善。

基于视觉转换的皮肤病生成与分类

LesionAid: Vision Transformers-based Skin Lesion Generation and Classification

Recently, Vision Transformers (ViTs) have shown competitive performance on
image recognition while requiring less vision-specific inductive biases. In
this paper, we investigate if such observation can be extended to image
generation. To this end, we integrate the ViT architecture into generative
adversarial networks (GANs). We observe that existing regularization methods
for GANs interact poorly with self-attention, causing serious instability
during training. To resolve this issue, we introduce novel regularization
techniques for training GANs with ViTs. Empirically, our approach, named
ViTGAN, achieves comparable performance to state-of-the-art CNN-based StyleGAN2
on CIFAR-10, CelebA, and LSUN bedroom datasets.

本文研究在生成对抗网络中引入 Vision Transformers (ViTs) 架构，并通过引入创新的正则化技术（ViTGAN）解决现有正则化方法与自注意力交互不良的问题，实验表明 ViTGAN 在 CIFAR-10，CelebA 和 LSUN 卧室数据集上表现不亚于基于卷积神经网络的 StyleGAN2 的最新成果。