Clustering data is an unsupervised learning approach that aims to divide a
set of data points into multiple groups. It is a crucial yet demanding subject
in machine learning and data mining. Its successful applications span various
fields. However, conventional clustering techniques necessitate the
consideration of balance significance in specific applications. Therefore, this
paper addresses the challenge of imbalanced clustering problems and presents a
new method for balanced clustering by utilizing entropy-aware similarity, which
can be defined as the degree of balances. We have coined the term,
entropy-aware similarity for balanced clustering (EASB), which maximizes
balance during clustering by complementary clustering of unbalanced data and
incorporating entropy in a novel similarity formula that accounts for both
angular differences and distances. The effectiveness of the proposed approach
is evaluated on actual melanoma medial data, specifically the International
Skin Imaging Collaboration (ISIC) 2019 and 2020 challenge datasets, to
demonstrate how it can successfully cluster the data while preserving balance.
Lastly, we can confirm that the proposed method exhibited outstanding
performance in detecting melanoma, comparing to classical methods.

本文提出了一种新的基于 entropy-aware similarity 的方法用于 balanced clustering, 通过将熵纳入一个新的相似度公式中，将不平衡的数据进行互补聚类来最大化平衡度，实现成功聚类和患者黑色素瘤的识别。

熵感知的相似度用于平衡聚类：以黑素瘤检测为例

Entropy-Aware Similarity for Balanced Clustering: A Case Study with  Melanoma Detection

We propose a novel type of balanced clustering algorithm to approximate
attention. Attention complexity is reduced from $O(N^2)$ to $O(N \log N)$,
where $N$ is the sequence length. Our algorithm, SMYRF, uses Locality Sensitive
Hashing (LSH) in a novel way by defining new Asymmetric transformations and an
adaptive scheme that produces balanced clusters. The biggest advantage of SMYRF
is that it can be used as a drop-in replacement for dense attention layers
without any retraining. On the contrary, prior fast attention methods impose
constraints (e.g. queries and keys share the same vector representations) and
require re-training from scratch. We apply our method to pre-trained
state-of-the-art Natural Language Processing and Computer Vision models and we
report significant memory and speed benefits. Notably, SMYRF-BERT outperforms
(slightly) BERT on GLUE, while using $50\%$ less memory. We also show that
SMYRF can be used interchangeably with dense attention before and after
training. Finally, we use SMYRF to train GANs with attention in high
resolutions. Using a single TPU, we were able to scale attention to 128x128=16k
and 256x256=65k tokens on BigGAN on CelebA-HQ.

我们提出了一种新型的平衡聚类算法 SMYRF，通过使用局部敏感哈希算法和一系列新异构变换，实现了由 O（N ^ 2）到 O（N log N）的注意力复杂度的有效减少，并在不需要重新训练的情况下拥有良好的性能表现。