Effective image classification hinges on discerning relevant features from
both foreground and background ele- ments, with the foreground typically
holding the critical informa- tion. While humans adeptly classify images with
limited exposure, artificial neural networks often struggle with feature
selection from rare samples. To address this challenge, we propose a novel
method for selecting class-relevant patch embeddings. Our ap- proach involves
splitting support and query images into patches, encoding them using a
pre-trained Vision Transformer (ViT) to obtain class embeddings and patch
embeddings, respectively. Subsequently, we filter patch embeddings using class
embeddings to retain only the class-relevant ones. For each image, we calculate
the similarity between class embedding and each patch embed- ding, sort the
similarity sequence in descending order, and only retain top-ranked patch
embeddings. By prioritizing similarity between the class embedding and patch
embeddings, we select top-ranked patch embeddings to be fused with class
embedding to form a comprehensive image representation, enhancing pattern
recognition across instances. Our strategy effectively mitigates the impact of
class-irrelevant patch embeddings, yielding improved performance in pre-trained
models. Extensive experiments on popular few-shot classification benchmarks
demonstrate the sim- plicity, efficacy, and computational efficiency of our
approach, outperforming state-of-the-art baselines under both 5-shot and 1-shot
scenarios.

我们提出了一种有效的图像分类方法，通过从前景和背景元素中选择相关特征和使用预训练的 Vision Transformer (ViT) 对图像进行编码，然后通过筛选类相关的补丁嵌入来改善预训练模型的性能，并在流行的少样本分类基准测试中超过了现有的基线。

少样本图像分类的类相关补丁嵌入选择

Class-relevant Patch Embedding Selection for Few-Shot Image  Classification

Assessing microsatellite stability status of a patient's colorectal cancer is
crucial in personalizing treatment regime. Recently,
convolutional-neural-networks (CNN) combined with transfer-learning approaches
were proposed to circumvent traditional laboratory testing for determining
microsatellite status from hematoxylin and eosin stained biopsy whole slide
images (WSI). However, the high resolution of WSI practically prevent direct
classification of the entire WSI. Current approaches bypass the WSI high
resolution by first classifying small patches extracted from the WSI, and then
aggregating patch-level classification logits to deduce the patient-level
status. Such approaches limit the capacity to capture important information
which resides at the high resolution WSI data. We introduce an effective
approach to leverage WSI high resolution information by momentum contrastive
learning of patch embeddings along with training a patient-level classifier on
groups of those embeddings. Our approach achieves up to 7.4\% better accuracy
compared to the straightforward patch-level classification and patient level
aggregation approach with a higher stability (AUC, $0.91 \pm 0.01$ vs. $0.85
\pm 0.04$, p-value$<0.01$). Our code can be found at
this https URL.

通过动量对比学习补充卷积神经网络、传输学习提出的小块级别分类和患者级别聚合的模式，从而更精确的评估患者结直肠癌的微卫星稳定状态

基于 Momentum 对比学习和组补丁嵌入的全幻灯片图像患者级微卫星稳定性评估

Patient-level Microsatellite Stability Assessment from Whole Slide Images By Combining Momentum Contrast Learning and Group Patch Embeddings

Extracting robust feature representation is one of the key challenges in
object re-identification (ReID). Although convolution neural network
(CNN)-based methods have achieved great success, they only process one local
neighborhood at a time and suffer from information loss on details caused by
convolution and downsampling operators (e.g. pooling and strided convolution).
To overcome these limitations, we propose a pure transformer-based object ReID
framework named TransReID. Specifically, we first encode an image as a sequence
of patches and build a transformer-based strong baseline with a few critical
improvements, which achieves competitive results on several ReID benchmarks
with CNN-based methods. To further enhance the robust feature learning in the
context of transformers, two novel modules are carefully designed. (i) The
jigsaw patch module (JPM) is proposed to rearrange the patch embeddings via
shift and patch shuffle operations which generates robust features with
improved discrimination ability and more diversified coverage. (ii) The side
information embeddings (SIE) is introduced to mitigate feature bias towards
camera/view variations by plugging in learnable embeddings to incorporate these
non-visual clues. To the best of our knowledge, this is the first work to adopt
a pure transformer for ReID research. Experimental results of TransReID are
superior promising, which achieve state-of-the-art performance on both person
and vehicle ReID benchmarks.

本文介绍了基于 transformer 的物体 ReID 方法 TransReID，通过重新排列 patch embeddings，设计新颖的 jigsaw patch module 和 side information embeddings 模块来提高对细节、摄像机视角等因素的鲁棒性，并在多个 ReID 数据集上实现了最优结果。