Detecting breast lesion in videos is crucial for computer-aided diagnosis.
Existing video-based breast lesion detection approaches typically perform
temporal feature aggregation of deep backbone features based on the
self-attention operation. We argue that such a strategy struggles to
effectively perform deep feature aggregation and ignores the useful local
information. To tackle these issues, we propose a spatial-temporal deformable
attention based framework, named STNet. Our STNet introduces a spatial-temporal
deformable attention module to perform local spatial-temporal feature fusion.
The spatial-temporal deformable attention module enables deep feature
aggregation in each stage of both encoder and decoder. To further accelerate
the detection speed, we introduce an encoder feature shuffle strategy for
multi-frame prediction during inference. In our encoder feature shuffle
strategy, we share the backbone and encoder features, and shuffle encoder
features for decoder to generate the predictions of multiple frames. The
experiments on the public breast lesion ultrasound video dataset show that our
STNet obtains a state-of-the-art detection performance, while operating twice
as fast inference speed. The code and model are available at
this https URL

通过引入空间 - 时间变形注意力模块实现局部空间 - 时间特征融合，本研究提出的 STNet 在每个编码器和解码器阶段都能有效地进行深度特征聚合，同时引入编码器特征洗牌策略提高多帧预测的检测速度。实验结果显示，STNet 在公共乳腺病变超声视频数据集上达到最先进的检测性能，同时有两倍的推理速度。