Traditional deep learning methods struggle to simultaneously segment,
recognize, and forecast human activities from sensor data. This limits their
usefulness in many fields such as healthcare and assisted living, where
real-time understanding of ongoing and upcoming activities is crucial. This
paper introduces P2LHAP, a novel Patch-to-Label Seq2Seq framework that tackles
all three tasks in a efficient single-task model. P2LHAP divides sensor data
streams into a sequence of "patches", served as input tokens, and outputs a
sequence of patch-level activity labels including the predicted future
activities. A unique smoothing technique based on surrounding patch labels, is
proposed to identify activity boundaries accurately. Additionally, P2LHAP
learns patch-level representation by sensor signal channel-independent
Transformer encoders and decoders. All channels share embedding and Transformer
weights across all sequences. Evaluated on three public datasets, P2LHAP
significantly outperforms the state-of-the-art in all three tasks,
demonstrating its effectiveness and potential for real-world applications.

传统深度学习方法难以从传感器数据中同时分割、识别和预测人类活动，限制了其在医疗保健和辅助生活等领域的实时理解能力。本文介绍了 P2LHAP，一种新颖的 Patch-to-Label Seq2Seq 框架，有效地解决了这三个任务，将传感器数据流划分为一系列 “补丁” 作为输入，输出包括预测未来活动的补丁级活动标签序列。作者提出了一种基于周围补丁标签的独特平滑技术，用于准确识别活动边界。此外，P2LHAP 通过传感器信号通道独立的 Transformer 编码器和解码器学习补丁级表示，所有通道在所有序列中共享嵌入和 Transformer 权重。经过三个公开数据集的评估，P2LHAP 在这三个任务中显著优于现有技术水平，展示了其在实际应用中的有效性和潜力。

P2LHAP: 基于可穿戴传感器的人体活动识别、分割和预测的 Patch-to-Label Seq2Seq Transformer

P2LHAP:Wearable sensor-based human activity recognition, segmentation  and forecast through Patch-to-Label Seq2Seq Transformer

Simultaneous translation involves translating a sentence before the speaker's
utterance is completed in order to realize real-time understanding in multiple
languages. This task is significantly more challenging than the general full
sentence translation because of the shortage of input information during
decoding. To alleviate this shortage, we propose multimodal simultaneous neural
machine translation (MSNMT), which leverages visual information as an
additional modality. Our experiments with the Multi30k dataset showed that
MSNMT significantly outperforms its text-only counterpart in more timely
translation situations with low latency. Furthermore, we verified the
importance of visual information during decoding by performing an adversarial
evaluation of MSNMT, where we studied how models behaved with incongruent input
modality and analyzed the effect of different word order between source and
target languages.

提出了一种多模式神经机器翻译方法，即多模式同时翻译 (MSNMT)，能够在翻译过程中利用视频信息和实现实时多语言理解。实验证实，该方法可显著提高翻译效率，特别是在低延迟情况下，比单纯文本的模型更加优秀。此外，我们通过对 MSNMT 的对抗性评估研究发现了视频信息在解码过程中的重要性，并分析了源语言和目标语言之间不同的单词顺序对翻译结果的影响。