Segment anything model (SAM) has achieved great success in the field of
natural image segmentation. Nevertheless, SAM tends to classify shadows as
background, resulting in poor segmentation performance for shadow detection
task. In this paper, we propose an simple but effective approach for fine
tuning SAM to detect shadows. Additionally, we also combine it with long
short-term attention mechanism to extend its capabilities to video shadow
detection. Specifically, we first fine tune SAM by utilizing shadow data
combined with sparse prompts and apply the fine-tuned model to detect a
specific frame (e.g., first frame) in the video with a little user assistance.
Subsequently, using the detected frame as a reference, we employ a long
short-term network to learn spatial correlations between distant frames and
temporal consistency between contiguous frames, thereby achieving shadow
information propagation across frames. Extensive experimental results
demonstrate that our method outperforms the state-of-the-art techniques, with
improvements of 17.2% and 3.3% in terms of MAE and IoU, respectively,
validating the effectiveness of our method.

本研究使用少量的用户协助，利用阴影数据和稀疏提示对 Segment anything model 进行微调，再结合长短期注意机制扩展其能力，使其可以用于视频阴影检测，与现有技术相比，实验结果表明，该方法在 MAE 和 IoU 方面分别提高 17.2% 和 3.3%，验证了我们提出方法的有效性。

检测任意阴影：视频阴影检测中的物体分割

Detect Any Shadow: Segment Anything for Video Shadow Detection

In this report we describe the technical details of our submission to the
EPIC-Kitchens Action Recognition 2020 Challenge. To participate in the
challenge we deployed spatio-temporal feature extraction and aggregation models
we have developed recently: Gate-Shift Module (GSM) [1] and EgoACO, an
extension of Long Short-Term Attention (LSTA) [2]. We design an ensemble of GSM
and EgoACO model families with different backbones and pre-training to generate
the prediction scores. Our submission, visible on the public leaderboard with
team name FBK-HUPBA, achieved a top-1 action recognition accuracy of 40.0% on
S1 setting, and 25.71% on S2 setting, using only RGB.

本文介绍了我们参加 EPIC-Kitchens 2020 挑战赛所使用的技术细节，我们使用 Gate-Shift 模块和 EgoACO 来产生预测分数，最终我们的提交记录在公共排行榜上取得了 40.0％的 top-1 动作识别精度，使用了仅 RGB 的 S1 设置和 25.71％的 S2 设置。