Foundation models have rapidly evolved and have achieved significant
accomplishments in computer vision tasks. Specifically, the prompt mechanism
conveniently allows users to integrate image prior information into the model,
making it possible to apply models without any training. Therefore, we propose
a method based on foundation models and zero training to solve the tasks of
photoacoustic (PA) image segmentation. We employed the segment anything model
(SAM) by setting simple prompts and integrating the model's outputs with prior
knowledge of the imaged objects to accomplish various tasks, including: (1)
removing the skin signal in three-dimensional PA image rendering; (2) dual
speed-of-sound reconstruction, and (3) segmentation of finger blood vessels.
Through these demonstrations, we have concluded that deep learning can be
directly applied in PA imaging without the requirement for network design and
training. This potentially allows for a hands-on, convenient approach to
achieving efficient and accurate segmentation of PA images. This letter serves
as a comprehensive tutorial, facilitating the mastery of the technique through
the provision of code and sample datasets.

该研究论文提出了一种基于基础模型和零训练的方法来解决光声图像分割的任务，通过设置简单提示和将模型输出与图像对象的先验知识相结合，实现了多种任务，包括三维光声图像去除皮肤信号、双声速重建以及手指血管分割。研究结果表明，深度学习可直接应用于光声成像，无需网络设计和训练，从而实现对光声图像的高效准确分割。该论文提供了代码和样本数据集，为技术的掌握提供了全面的指导。

基于基础模型的简化光声图像处理：一种无需训练的解决方案

Streamlined Photoacoustic Image Processing with Foundation Models: A  Training-Free Solution

Recent research has demonstrated the efficacy of pre-training graph neural
networks (GNNs) to capture the transferable graph semantics and enhance the
performance of various downstream tasks. However, the semantic knowledge
learned from pretext tasks might be unrelated to the downstream task, leading
to a semantic gap that limits the application of graph pre-training. To reduce
this gap, traditional approaches propose hybrid pre-training to combine various
pretext tasks together in a multi-task learning fashion and learn multi-grained
knowledge, which, however, cannot distinguish tasks and results in some
transferable task-specific knowledge distortion by each other. Moreover, most
GNNs cannot distinguish nodes located in different parts of the graph, making
them fail to learn position-specific knowledge and lead to suboptimal
performance. In this work, inspired by the prompt-based tuning in natural
language processing, we propose a unified framework for graph hybrid
pre-training which injects the task identification and position identification
into GNNs through a prompt mechanism, namely multi-task graph dual prompt
(ULTRA-DP). Based on this framework, we propose a prompt-based transferability
test to find the most relevant pretext task in order to reduce the semantic
gap. To implement the hybrid pre-training tasks, beyond the classical edge
prediction task (node-node level), we further propose a novel pre-training
paradigm based on a group of $k$-nearest neighbors (node-group level). The
combination of them across different scales is able to comprehensively express
more structural semantics and derive richer multi-grained knowledge. Extensive
experiments show that our proposed ULTRA-DP can significantly enhance the
performance of hybrid pre-training methods and show the generalizability to
other pre-training tasks and backbone architectures.

本文提出了一种基于图混合预训练的统一框架，通过提示机制将任务识别和位置识别注入图神经网络，从而有效减小语义差距，实现更好的性能提升。

ULTRA-DP：将图预训练与多任务图双提示统一化

ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt

3D object detection has achieved significant performance in many fields,
e.g., robotics system, autonomous driving, and augmented reality. However, most
existing methods could cause catastrophic forgetting of old classes when
performing on the class-incremental scenarios. Meanwhile, the current
class-incremental 3D object detection methods neglect the relationships between
the object localization information and category semantic information and
assume all the knowledge of old model is reliable. To address the above
challenge, we present a novel Incremental 3D Object Detection framework with
the guidance of prompting, i.e., I3DOD. Specifically, we propose a task-shared
prompts mechanism to learn the matching relationships between the object
localization information and category semantic information. After training on
the current task, these prompts will be stored in our prompt pool, and perform
the relationship of old classes in the next task. Moreover, we design a
reliable distillation strategy to transfer knowledge from two aspects: a
reliable dynamic distillation is developed to filter out the negative knowledge
and transfer the reliable 3D knowledge to new detection model; the relation
feature is proposed to capture the responses relation in feature space and
protect plasticity of the model when learning novel 3D classes. To the end, we
conduct comprehensive experiments on two benchmark datasets and our method
outperforms the state-of-the-art object detection methods by 0.6% - 2.7% in
terms of mAP@0.25.

我们提出了一种新的增量 3D 物体检测框架，采用提示引导的方式，通过任务共享的提示机制学习目标定位信息和类别语义信息之间的匹配关系，并通过可靠的蒸馏策略从可靠的模型转移知识，最终在两个基准数据集上表现出优于现有方法的检测性能。