Neuroscientific research has revealed that the complex brain network can be
organized into distinct functional communities, each characterized by a
cohesive group of regions of interest (ROIs) with strong interconnections.
These communities play a crucial role in comprehending the functional
organization of the brain and its implications for neurological conditions,
including Autism Spectrum Disorder (ASD) and biological differences, such as in
gender. Traditional models have been constrained by the necessity of predefined
community clusters, limiting their flexibility and adaptability in deciphering
the brain's functional organization. Furthermore, these models were restricted
by a fixed number of communities, hindering their ability to accurately
represent the brain's dynamic nature. In this study, we present a token
clustering brain transformer-based model ($\texttt{TC-BrainTF}$) for joint
community clustering and classification. Our approach proposes a novel token
clustering (TC) module based on the transformer architecture, which utilizes
learnable prompt tokens with orthogonal loss where each ROI embedding is
projected onto the prompt embedding space, effectively clustering ROIs into
communities and reducing the dimensions of the node representation via merging
with communities. Our results demonstrate that our learnable community-aware
model $\texttt{TC-BrainTF}$ offers improved accuracy in identifying ASD and
classifying genders through rigorous testing on ABIDE and HCP datasets.
Additionally, the qualitative analysis on $\texttt{TC-BrainTF}$ has
demonstrated the effectiveness of the designed TC module and its relevance to
neuroscience interpretations.

通过神经科学研究，本文提出了一种基于 Transformer 架构的令牌聚类脑模型（TC-BrainTF）来联合社群聚类和分类，通过在神经考察数据集上进行严格测试，显示出了 TC-BrainTF 模型在识别自闭症谱系障碍（ASD）和分类性别方面的准确性提高，并展示了所设计的 TC 模块的效果和对神经科学解释的相关性。

学习可学习社区感知的大脑连接组分析的变压器与令牌聚类

Learnable Community-Aware Transformer for Brain Connectome Analysis with  Token Clustering

Mobile monocular 3D object detection (Mono3D) (e.g., on a vehicle, a drone,
or a robot) is an important yet challenging task. Existing transformer-based
offline Mono3D models adopt grid-based vision tokens, which is suboptimal when
using coarse tokens due to the limited available computational power. In this
paper, we propose an online Mono3D framework, called MonoATT, which leverages a
novel vision transformer with heterogeneous tokens of varying shapes and sizes
to facilitate mobile Mono3D. The core idea of MonoATT is to adaptively assign
finer tokens to areas of more significance before utilizing a transformer to
enhance Mono3D. To this end, we first use prior knowledge to design a scoring
network for selecting the most important areas of the image, and then propose a
token clustering and merging network with an attention mechanism to gradually
merge tokens around the selected areas in multiple stages. Finally, a
pixel-level feature map is reconstructed from heterogeneous tokens before
employing a SOTA Mono3D detector as the underlying detection core. Experiment
results on the real-world KITTI dataset demonstrate that MonoATT can
effectively improve the Mono3D accuracy for both near and far objects and
guarantee low latency. MonoATT yields the best performance compared with the
state-of-the-art methods by a large margin and is ranked number one on the
KITTI 3D benchmark.

本文提出一种名为 MonoATT 的在线移动单眼 3D 物体检测框架，该框架采用形状和大小不同的异构令牌进行识别，并结合注意力机制和分数网络提高检测准确率，实验证明 MonoATT 在 KITTI 3D 基准上排名第一。

MonoATT：自适应令牌变换的在线单目三维物体检测

MonoATT: Online Monocular 3D Object Detection with Adaptive Token  Transformer

Recently, large-scale pre-training methods like CLIP have made great progress
in multi-modal research such as text-video retrieval. In CLIP, transformers are
vital for modeling complex multi-modal relations. However, in the vision
transformer of CLIP, the essential visual tokenization process, which produces
discrete visual token sequences, generates many homogeneous tokens due to the
redundancy nature of consecutive and similar frames in videos. This
significantly increases computation costs and hinders the deployment of video
retrieval models in web applications. In this paper, to reduce the number of
redundant video tokens, we design a multi-segment token clustering algorithm to
find the most representative tokens and drop the non-essential ones. As the
frame redundancy occurs mostly in consecutive frames, we divide videos into
multiple segments and conduct segment-level clustering. Center tokens from each
segment are later concatenated into a new sequence, while their original
spatial-temporal relations are well maintained. We instantiate two clustering
algorithms to efficiently find deterministic medoids and iteratively partition
groups in high dimensional space. Through this token clustering and center
selection procedure, we successfully reduce computation costs by removing
redundant visual tokens. This method further enhances segment-level semantic
alignment between video and text representations, enforcing the spatio-temporal
interactions of tokens from within-segment frames. Our method, coined as
CenterCLIP, surpasses existing state-of-the-art by a large margin on typical
text-video benchmarks, while reducing the training memory cost by 35\% and
accelerating the inference speed by 14\% at the best case. The code is
available at
\href{{https://github.com/mzhaoshuai/CenterCLIP}}{{this https URL}}.

本文介绍了 CenterCLIP，它是一种新的视频标记聚类算法，在减少多余的视频标记、提高视频与文本表示的语义对齐方面表现优异，并取得了比现有最先进方法更好的结果