Multi-modal human action segmentation is a critical and challenging task with
a wide range of applications. Nowadays, the majority of approaches concentrate
on the fusion of dense signals (i.e., RGB, optical flow, and depth maps).
However, the potential contributions of sparse IoT sensor signals, which can be
crucial for achieving accurate recognition, have not been fully explored. To
make up for this, we introduce a Sparse signalguided Transformer (SigFormer) to
combine both dense and sparse signals. We employ mask attention to fuse
localized features by constraining cross-attention within the regions where
sparse signals are valid. However, since sparse signals are discrete, they lack
sufficient information about the temporal action boundaries. Therefore, in
SigFormer, we propose to emphasize the boundary information at two stages to
alleviate this problem. In the first feature extraction stage, we introduce an
intermediate bottleneck module to jointly learn both category and boundary
features of each dense modality through the inner loss functions. After the
fusion of dense modalities and sparse signals, we then devise a two-branch
architecture that explicitly models the interrelationship between action
category and temporal boundary. Experimental results demonstrate that SigFormer
outperforms the state-of-the-art approaches on a multi-modal action
segmentation dataset from real industrial environments, reaching an outstanding
F1 score of 0.958. The codes and pre-trained models have been available at
this https URL

介绍了一种稀疏信号引导的变压器模型 (SigFormer)，该模型通过融合密集信号和稀疏信号来提高多模态人体动作分割的准确性，在处理稀疏信号时使用了蒙版注意力机制，同时在提取特征和建模动作类别与时间边界关系时强调了边界信息。在实验中，SigFormer 在真实工业环境下的多模态动作分割数据集上表现优秀，F1 得分达到 0.958。

SigFormer: 稀疏信号引导变换器用于多模态人体动作分割

SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action  Segmentation

Deep hedging is a promising direction in quantitative finance, incorporating
models and techniques from deep learning research. While giving excellent
hedging strategies, models inherently requires careful treatment in designing
architectures for neural networks. To mitigate such difficulties, we introduce
SigFormer, a novel deep learning model that combines the power of path
signatures and transformers to handle sequential data, particularly in cases
with irregularities. Path signatures effectively capture complex data patterns,
while transformers provide superior sequential attention. Our proposed model is
empirically compared to existing methods on synthetic data, showcasing faster
learning and enhanced robustness, especially in the presence of irregular
underlying price data. Additionally, we validate our model performance through
a real-world backtest on hedging the SP 500 index, demonstrating positive
outcomes.

SigFormer 是一种深度学习模型，结合了路径签名和 Transformer 的优点，用于处理具有不规律性的序列数据，特别适用于金融领域中的对冲策略设计。模型在合成数据上的实证比较展示了其更快的学习速度和增强的稳健性，尤其是在存在不规律的基础价格数据的情况下，同时通过对标普 500 指数的实际对冲回测验证了模型的性能。